cortoproject / corto

A hierarchical object store for connecting realtime machine data with web applications, historians & more
https://www.corto.io
MIT License
87 stars 14 forks source link

Add low-level store API that allows for more flexibility & better performance #635

Closed SanderMertens closed 6 years ago

SanderMertens commented 6 years ago

In some cases the regular store API (corto_declareChild, corto_define etc) is not flexible enough, or performs actions that in some contexts are redundant. To enable applications to do more powerful things with the store, and optimize certain scenarios, a lower-level API call is needed.

Examples of redundancy are:

Additionally, the sequence of default store operations can become quite complex in multi-threaded scenarios where multiple threads can be instantiating the same object at the same time, values have to be assigned, and objects have to be either looked up or created.

The current standard API is already a thin layer upon a smaller, more powerful (but also more complex) API. A new function should be added that wraps around the internal APIs, and allows for executing multiple low-level operations in the correct sequence.

The API could look like this:

typedef enum corto_kind {
    CORTO_DO_DECLARE = 0x1, // Find or declare an object. If not specified, function does a lookup
    CORTO_DO_RECURSIVE_DECLARE = 0x3, // Allow for creation with recursive identifiers (foo/bar)
    CORTO_DO_DEFINE = 0x4, // Define object
    CORTO_DO_UPDATE = 0x8, // Update object
    CORTO_DO_ORPHAN = 0x10, // Create orphaned object
    CORTO_DO_RESUME = 0x20, // Resume object
    CORTO_DO_FORCE_TYPE = 0x40, // Ensure object is of the specified type (when found)
    CORTO_DO_LOOKUP_TYPE = 0x80 // Use rules for looking up type identifiers
} corto_kind;

corto_object corto(
    corto_object parent, // Parent
    const char *id, // Identifier
    corto_type type, // Type
    corto_object ref, // Object reference (optional)
    corto_contentType contentType, // Content type handle
    void *value, // Pointer to value
    corto_attr attrs, // Object attributes
    corto_kind kind); // Operations to be executed

For example, an application that needs to either find or create an object in a multithreaded context, and define an object when its created, and update an object when its found, could be implemented with a single function call:

corto(parent, id, type, NULL, contentTypeHandle, valuePtr, CORTO_ATTR_DEFAULT,
    CORTO_DO_DECLARE | CORTO_DO_UPDATE | CORTO_DO_DEFINE);

Because the interface is quite complex, the API should initially be marked as experimental to allow for improvements that are backwards incompatible

SanderMertens commented 6 years ago

With this function implemented, some other less often used functions can be deprecated to reduce the size of the API. Functions that could be deprecated are:

corto_declareOrphan
corto_createOrphan
corto_findOrDeclare
corto_findOrCreate
corto_find
corto_fromcontent
corto_object_fromcontent
SanderMertens commented 6 years ago

This function has been changed in the v2.0 API upgrade, so that it becomes more readable, and more arguments can be added in the future without breaking backwards compatibility. An example:

corto_object foo = corto(CORTO_DECLARE, {.parent = root_o, .id = "foo"});

To enable this syntax, the function accepts an argument of a struct-type that contains all the possible arguments of the function. This enables arguments to be referenced by name. The function itself is now a macro which includes a cast to the corto_action type for the second parameter, so that the fully expanded function call looks like this:

corto_object foo = corto(CORTO_DECLARE, (corto_action){.parent = root_o, .id = "foo"});