cortoproject / corto

A hierarchical object store for connecting realtime machine data with web applications, historians & more
https://www.corto.io
MIT License
86 stars 14 forks source link

Enable selecting/subscribing for unknown objects #671

Closed SanderMertens closed 6 years ago

SanderMertens commented 6 years ago

Unknown objects are placeholder objects in stores that require knowledge of the object, but do not store the object itself.

A typical example is that of a mount that provides access to foo/bar, but not to foo. In order to make foo/bar discoverable, it may serve up foo with type unknown to indicate that more objects exist on a deeper level. In the in-memory store, objects of an unknown type are automatically created when a user creates a child object for a parent that did not yet exist. For example:

corto_create(root_o, "data/foo/bar", corto_int32_o);

The data object already existed (it exists in every store), and is left unaltered. The data/foo object did not yet exist, and will be created as unknown. The data/foo/bar object will be created with type int32.

By default, operations that inspect the store or subscribe for updates do not return unknown objects, because conceptually, nothing is known about their existence. In some cases however this is undesired. In particular, when generically discovering content in a store, unknown objects are required as without them, objects could be dangling.

To get unknown objects from corto_select or corto_subscribe, a new method called yield_unknown will be added, that can be used like this for corto_select:

corto_select("*").yield_unknown().iter(&iterator);

And like this for corto_subscribe:

corto_subscribe("*").yield_unknown().callback(on_event_cb);