Library Setup

typedef struct fluxEngine_C_v1_Handle fluxEngine_C_v1_Handle

fluxEngine Handle

This opaque data structure contains a handle to all fluxEngine functionality. It may be obtained via fluxEngine_C_v1_init() and should be destroyed by the user via fluxEngine_C_v1_destroy().

Currently only one handle may be created at a time. (This restriction will be relaxed in a future version.)

Each handle is associated with a number of processing threads that the user can manage. A handle may only be used to process a single model at the same time, though multiple models may be loaded for a given handle.

int fluxEngine_C_v1_init(void const *license_data, size_t license_data_size, fluxEngine_C_v1_Handle **handle, fluxEngine_C_v1_Error **error)

Initialize a fluxEngine handle.

Initializes fluxEngine. Valid license data must be passed to this function, otherwise fluxEngine will not initialize itself. It is up to the user to read the license data from the given license file, if they choose to store it in a file.

If the call is successful, a pointer to the newly created handle will be stored in handle. The following is the typical usage pattern of this method: 

fluxEngine_C_v1_Error* error = NULL;
fluxEngine_C_v1_Handle* handle = NULL;
int ret = fluxEngine_C_v1_init(license_data, license_data_size,
                               &handle, &error);
if (ret != 0) {
    // perform error handling
    // ...
    // cleanup error structure
    fluxEngine_C_v1_Error_free(error);
    // don't proceed
    return;
}
// handle is now valid
// at the end, when the handle is no longer needed
fluxEngine_C_v1_destroy(handle);
The following specific error codes may be returned by this function:

  • fluxEngine_C_v1_ErrorCode_Unknown

  • fluxEngine_C_v1_ErrorCode_AllocationFailure

  • fluxEngine_C_v1_ErrorCode_InvalidArgument

  • fluxEngine_C_v1_ErrorCode_HandleAlreadyCreated

  • fluxEngine_C_v1_ErrorCode_LicenseExpired

  • fluxEngine_C_v1_ErrorCode_LicenseWrongProduct

  • fluxEngine_C_v1_ErrorCode_LicenseUpdateExpired

  • fluxEngine_C_v1_ErrorCode_LicenseIdentifierMismatch

  • fluxEngine_C_v1_ErrorCode_InvalidLicense

Parameters

  • license_data – The raw bytes of the license

  • license_data_size – The number of raw bytes of the license

  • handle – [out] A pointer to the resulting handle, on success

  • error – [out] The resulting error object, if an error occurs. See the documentation of the fluxEngine_C_v1_Error structure for details on error handling.

Returns

0 on success, -1 on failure

int fluxEngine_C_v1_create_processing_threads(fluxEngine_C_v1_Handle *handle, int thread_count, fluxEngine_C_v1_Error **error)

Create processing threads.

fluxEngine may use parallization to speed up processing. In order to achieve this background threads must be created to run on additional CPU cores.

Calling this function is optional: by default processing will be single-threaded.

This function will start thread_count - 1 threads when called, as the thread that asks for processing is always considered to be the first thread (with id 0). For example, if 4 is supplied to thread_count this function will start 3 threads that run in the background. The thread that the user uses to call fluxEngine_C_v1_ProcessingContext_process_next() will be considered the thread with id 0, making processing use a total of 4 threads, which is the value supplied for thread_count.

This function may only be called if there are currently no background threads associated with the default processing queue set of the handle. Otherwise fluxEngine_C_v1_stop_processing_threads() must be called first to change the number of threads.

This function will act on the default processing queue set of the handle (that is always present), but will not affect any other processing queue sets that the user has created for the handle.

Any processing context that was created for the default processing queue set of the handle before a call to this function was made is marked as invalid and can only be destroyed, but not used anymore.

The following specific error codes may be returned by this function:

  • fluxEngine_C_v1_ErrorCode_Unknown

  • fluxEngine_C_v1_ErrorCode_AllocationFailure

  • fluxEngine_C_v1_ErrorCode_InvalidArgument

  • fluxEngine_C_v1_ErrorCode_HandleNoLongerValid

  • fluxEngine_C_v1_ErrorCode_ThreadCreationError

Parameters

  • handle – The handle to create the threads for

  • thread_count – The number of threads to use for parallel processing (one less than this number will be created by this function, see the description for details)

  • error – [out] The resulting error object, if an error occurs. See the documentation of the fluxEngine_C_v1_Error structure for details on error handling.

Returns

0 on success, -1 on failure

typedef int (*fluxEngine_C_v1_ThreadInitFunction)(void *context, int thread_id, int thread_count)

Thread initialization function.

This callback may be passed to fluxEngine_C_v1_create_processing_threads_ex(). It will be called at the beginning of the newly created threads and allows the user to change properties of those threads before processing starts.

For example, this may be used to update the thread’s priority or to update its CPU pinning.

This function must not block for a long period of time, as fluxEngine_C_v1_create_processing_threads_ex() will wait on the completion of this function before it returns.

Important: any call to any fluxEngine function that accesses the handle in question during this callback will result in a deadlock.

This callback must not throw any C++ exception, or use longjmp() from within; that behavior is undefined.

Param context

The context supplied as the init_function_context parameter. This may be used by the user to pass data into the callback.

Param thread_id

The id of the thread this is called for. 1 indicates the second thread, 2 the third, etc. Note that this function will never be called for the first thread, see the thread creation functions for details.

Param thread_count

The total number of threads being created.

Return

This callback should return 0 on success. Any other value will indicate a failure and cause the thread creation function to tear down the threads again and fail with an error itself.

int fluxEngine_C_v1_create_processing_threads_ex(fluxEngine_C_v1_Handle *handle, int thread_count, fluxEngine_C_v1_ThreadInitFunction init_function, void *init_function_context, fluxEngine_C_v1_Error **error)

Create processing threads (extended version)

Please read the documentation of fluxEngine_C_v1_create_processing_threads() for general details.

This extended function allows the user to supply a thread initialization function that will be called at the beginning of the newly created background threads. This allows the user to customize the thread properties (such as the thread priority or the CPU affinity) themselves.

This function will only return once all thread initialization functions have run.

The thread initialization functions are only called for the backgronud threads that are started by this function; this means that for a thread_count of 4 the initialization function will be called in 3 background threads, and it is up to the user to alter the thread in which they call fluxEngine_C_v1_ProcessingContext_process_next() to process data with fluxEngine.

The threads will be created sequentially, the next thread being created only after the previous thread’s initialization function has completed. This allows the user to directly modify global data structures in the initialization functions without the need for locking.

Important: any attempt to call function that accesses the default processing queue set of the handle (e.g. creating a processing context) inside the initialization functions will create a deadlock.

This function may only be called if there are currently no background threads associated with this handle. Otherwise fluxEngine_C_v1_stop_processing_threads() must be called first to change the number of threads.

This function will act on the default processing queue set of the handle (that is always present), but will not affect any other processing queue sets that the user has created for the handle.

Any processing context that was created for the default processing queue set of the handle before a call to this function was made is marked as invalid and can only be destroyed, but not used anymore.

The following specific error codes may be returned by this function:

  • fluxEngine_C_v1_ErrorCode_Unknown

  • fluxEngine_C_v1_ErrorCode_AllocationFailure

  • fluxEngine_C_v1_ErrorCode_InvalidArgument

  • fluxEngine_C_v1_ErrorCode_HandleNoLongerValid

  • fluxEngine_C_v1_ErrorCode_ThreadCreationError

  • fluxEngine_C_v1_ErrorCode_ThreadInitFunctionError

See also

fluxEngine_C_v1_create_processing_threads()

Parameters

  • handle – The handle to create the threads for

  • thread_count – The number of threads to use for parallel processing (one less than this number will be created by this function, see the description for details)

  • init_function – The initialization function to call at the start of each newly created background thread

  • init_function_context – An arbitrary context that will be passed to the initialization function

  • error – [out] The resulting error object, if an error occurs. See the documentation of the fluxEngine_C_v1_Error structure for details on error handling.

Returns

0 on success, -1 on failure

void fluxEngine_C_v1_stop_processing_threads(fluxEngine_C_v1_Handle *handle)

Stop background threads of a handle.

This will stop any background threads that are currently associated with the default processing queue set of a given handle. If processing is currently active on the default processnig queue set of the handle, it will be aborted, as if fluxEngine_C_v1_ProcessingContext_abort() had been called. In that case this method may take a bit of time, as abort operations are not immediate, and this method will wait until the abort has completed.

This function will act on the default processing queue set of the handle (that is always present), but will not affect any other processing queue sets that the user has created for the handle.

Any processing context that was created for the default processing queue set of the handle before a call to this function was made is marked as invalid and can only be destroyed, but not used anymore.

This method is always successful: the only errors that could occur when calling this method would be non-recoverable.

If NULL is passed to this method, it will do nothing.

Parameters

  • handle – The handle to stop the threads for

void fluxEngine_C_v1_destroy(fluxEngine_C_v1_Handle *handle)

Destroy a handle.

Destroy a library handle, freeing its resources. All background threads will be stopped in the same manner as if fluxEngine_C_v1_stop_processing_threads() had been called.

Any processing context associated with this handle will be marked as invalid and may hence not be used anymore. However, some memory associated with remaining processing contexts that have not been freed previous to a call to this method may still be in use until each remaining processing context is freed by the user.

If NULL is passed to this method, it will do nothing.

Parameters

  • handle – The handle to destroy

Processing Queue Sets

typedef struct fluxEngine_C_v1_ProcessingQueueSet fluxEngine_C_v1_ProcessingQueueSet

Processing Queue Set.

This opaque data structure wraps a processing queue set. It may be created via the fluxEngine_C_v1_ProcessingQueueSet_create() function and destroyed via the fluxEngine_C_v1_destroy_processing_queue_set() function.

A processing queue set is the basic building block that allows data to be processed. Only one thread may perform data processing with the same processing queue set at the same time; locking is used to ensure that this happens.

The user is free to create as many processing queue sets for a given handle as they wish. Processing via multiple processing queue sets may happen from different threads at the same time.

int fluxEngine_C_v1_ProcessingQueueSet_create(fluxEngine_C_v1_Handle *handle, fluxEngine_C_v1_ProcessingQueueSet **processing_queue_set, fluxEngine_C_v1_Error **error)

Create a new processing queue set.

Creates a new processing queue set for a given fluxEngine handle. This may then be used to create processing contexts.

If the call is successful, a pointer to the newly created processing queue set will be stored in processing_queue_set. The is the typical usage pattern of this method: 

fluxEngine_C_v1_Handle* handle = ...;
fluxEngine_C_v1_Error* error = NULL;
fluxEngine_C_v1_ProcessingQueueSet* processing_queue_set = NULL;
int ret = fluxEngine_C_v1_init(handle, &processing_queue_set,
                               &error);
if (ret != 0) {
    // perform error handling
    // ...
    // cleanup error structure
    fluxEngine_C_v1_Error_free(error);
    // don't proceed
    return;
}
// processing_queue_set is now valid
// ...
// at the end, when the processing queue set is no longer
// needed:
fluxEngine_C_v1_destroy_processing_queue_set(processing_queue_set);
The following specific error codes may be returned by this function:

  • fluxEngine_C_v1_ErrorCode_Unknown

  • fluxEngine_C_v1_ErrorCode_AllocationFailure

  • fluxEngine_C_v1_ErrorCode_InvalidArgument

Parameters

  • handle – The fluxEngine handle

  • processing_queue_set – [out] A pointer to the resulting processing queue set, on success

  • error – [out] The resulting error object, if an error occurs. See the documentation of the fluxEngine_C_v1_Error structure for details on error handling.

Returns

0 on success, -1 on failure

int fluxEngine_C_v1_ProcessingQueueSet_create_threads(fluxEngine_C_v1_ProcessingQueueSet *processing_queue_set, int thread_count, fluxEngine_C_v1_Error **error)

Create processing threads for a given processing queue set.

fluxEngine may use parallization to speed up processing. In order to achieve this background threads must be created to run on additional CPU cores.

Calling this function is optional: by default processing will be single-threaded.

This function will start thread_count - 1 threads when called, as the thread that asks for processing is always considered to be the first thread (with id 0). For example, if 4 is supplied to thread_count this function will start 3 threads that run in the background. The thread that the user uses to call fluxEngine_C_v1_ProcessingContext_process_next() will be considered the thread with id 0, making processing use a total of 4 threads, which is the value supplied for thread_count.

This function may only be called if there are currently no background threads associated with this processing queue set. Otherwise fluxEngine_C_v1_stop_processing_queue_set_threads() must be called first to change the number of threads.

Any processing context that was created for the same processing queue set before a call to this function was made is marked as invalid and can only be destroyed, but not used anymore.

The following specific error codes may be returned by this function:

  • fluxEngine_C_v1_ErrorCode_Unknown

  • fluxEngine_C_v1_ErrorCode_AllocationFailure

  • fluxEngine_C_v1_ErrorCode_InvalidArgument

  • fluxEngine_C_v1_ErrorCode_HandleNoLongerValid

  • fluxEngine_C_v1_ErrorCode_ThreadCreationError

Parameters

  • processing_queue_set – The processing queue set to create the threads for

  • thread_count – The number of threads to use for parallel processing (one less than this number will be created by this function, see the description for details)

  • error – [out] The resulting error object, if an error occurs. See the documentation of the fluxEngine_C_v1_Error structure for details on error handling.

Returns

0 on success, -1 on failure

int fluxEngine_C_v1_ProcessingQueueSet_create_threads_ex(fluxEngine_C_v1_ProcessingQueueSet *processing_queue_set, int thread_count, fluxEngine_C_v1_ThreadInitFunction init_function, void *init_function_context, fluxEngine_C_v1_Error **error)

Create processing threads for a given processing queue set (extended version)

Please read the documentation of fluxEngine_C_v1_ProcessingQueueSet_create_threads() for general details.

This extended function allows the user to supply a thread initialization function that will be called at the beginning of the newly created background threads. This allows the user to customize the thread properties (such as the thread priority or the CPU affinity) themselves.

This function will only return once all thread initialization functions have run.

The thread initialization functions are only called for the backgronud threads that are started by this function; this means that for a thread_count of 4 the initialization function will be called in 3 background threads, and it is up to the user to alter the thread in which they call fluxEngine_C_v1_ProcessingContext_process_next() to process data with fluxEngine.

The threads will be created sequentially, the next thread being created only after the previous thread’s initialization function has completed. This allows the user to directly modify global data structures in the initialization functions without the need for locking.

Important: any attempt to call a function that accesses this processing queue set inside the initialization functions will create a deadlock.

This function may only be called if there are currently no background threads associated with this processing queue set. Otherwise fluxEngine_C_v1_stop_processing_queue_set_threads() must be called first to change the number of threads.

Any processing context that was created for the same processing queue set before a call to this function was made is marked as invalid and can only be destroyed, but not used anymore.

The following specific error codes may be returned by this function:

  • fluxEngine_C_v1_ErrorCode_Unknown

  • fluxEngine_C_v1_ErrorCode_AllocationFailure

  • fluxEngine_C_v1_ErrorCode_InvalidArgument

  • fluxEngine_C_v1_ErrorCode_HandleNoLongerValid

  • fluxEngine_C_v1_ErrorCode_ThreadCreationError

  • fluxEngine_C_v1_ErrorCode_ThreadInitFunctionError

See also

fluxEngine_C_v1_ProcessingQueueSet_create_threads()

Parameters

  • processing_queue_set – The processing queue set to create the threads for

  • thread_count – The number of threads to use for parallel processing (one less than this number will be created by this function, see the description for details)

  • init_function – The initialization function to call at the start of each newly created background thread

  • init_function_context – An arbitrary context that will be passed to the initialization function

  • error – [out] The resulting error object, if an error occurs. See the documentation of the fluxEngine_C_v1_Error structure for details on error handling.

Returns

0 on success, -1 on failure

void fluxEngine_C_v1_ProcessingQueueSet_stop_threads(fluxEngine_C_v1_ProcessingQueueSet *processing_queue_set)

Stop background threads of a processing queue set.

This will stop any background threads that are currently associated with a given processing queue set. If processing is currently active on the handle, it will be aborted, as if fluxEngine_C_v1_ProcessingContext_abort() had been called. In that case this method may take a bit of time, as abort operations are not immediate, and this method will wait until the abort has completed.

Any processing context that was created for the same processing queue set before a call to this function was made is marked as invalid and can only be destroyed, but not used anymore.

This method is always successful: the only errors that could occur when calling this method would be non-recoverable.

If NULL is passed to this method, it will do nothing.

Parameters

  • processing_queue_set – The processing queue set to stop the threads for

void fluxEngine_C_v1_ProcessingQueueSet_free(fluxEngine_C_v1_ProcessingQueueSet *processing_queue_set)

Destroy a processing queue set.

Destroy a processing queue set, freeing its resources. All background threads will be stopped in the same manner as if fluxEngine_C_v1_ProcessingQueueSet_stop_threads() had been called.

Any processing context associated with this processing queue set will be marked as invalid and may hence not be used anymore. However, some memory associated with remaining processing contexts that have not been freed previous to a call to this method may still be in use until each remaining processing context is freed by the user.

All other processing queue sets of the handle will remain valid.

If NULL is passed to this method, it will do nothing.

Parameters

  • processing_queue_set – The processing queue set to destroy