The Data Broker (DBR) is a distributed, in-memory container of key-value stores enabling applications in a workflow to exchange data through one or more shared namespaces. Thanks to a small set of primitives, applications in a workflow deployed in a (possibly) shared nothing distributed cluster, can easily share and exchange data and messages with a minimum effort. In- spired by the Linda coordination and communication model, the Data Broker provides a unified shared namespace to applications, which is independent from applications’ programming and communication model.
This PR introduces an initial iterator feature (and API) to the Data Broker. It uses only a single API call to work with iterators.
DBR_Iterator_t iterator = DBR_ITERATOR_NEW;
do {
iterator = dbrIterator( namespace, iterator, group, filter, keybuffer );
/* do something with key in keybuffer */
} while( iterator != DBR_ITERATOR_DONE )
with current_iterator = NULL you create a new iterator;
if new_iterator is returned as NULL, the iteration reached the end
keybuffer is filled with the next available key
internally it uses caching/prefetching so that ~10% of the calls will complete immediately without back-end storage interaction
Cleanup of iterators happens only if iteration is done until the end is reached, otherwise there's a limit of 10 in-progress iterators. If that limit is exceeded, this initial implementation will not be able to create new iterators
This is the first version of the API, suggestions are welcome. Depending on feedback, there maybe minor changes in the future.
This PR introduces an initial iterator feature (and API) to the Data Broker. It uses only a single API call to work with iterators.
This is the first version of the API, suggestions are welcome. Depending on feedback, there maybe minor changes in the future.