search

IBM / data-broker

The Data Broker (DBR) is a distributed, in-memory container of key-value stores enabling applications in a workflow to exchange data through one or more shared namespaces. Thanks to a small set of primitives, applications in a workflow deployed in a (possibly) shared nothing distributed cluster, can easily share and exchange data and messages with a minimum effort. In- spired by the Linda coordination and communication model, the Data Broker provides a unified shared namespace to applications, which is independent from applications’ programming and communication model.
Apache License 2.0
19 stars 23 forks source link

Process hangs when calling directory with empty match_template #130

Closed jmoon1506 closed 3 years ago

jmoon1506 commented 4 years ago 
from dbr_module import dbr
from _dbr_interface import ffi

dbr_hdl = dbr.create("dummy_namespace", dbr.DBR_PERST_VOLATILE_SIMPLE, ffi.new('DBR_GroupList_t'))

dbr.put(dbr_hdl, 100, 'dummy_key_1', dbr.DBR_GROUP_EMPTY)

# OKAY
keys, nKeys, res = dbr.directory(dbr_hdl, 'dummy_key_*', dbr.DBR_GROUP_EMPTY, 1000, 1000)
print(keys)

# HANGS FOREVER
keys, nKeys, res = dbr.directory(dbr_hdl, 'not_dummy_key_*', dbr.DBR_GROUP_EMPTY, 1000, 1000)
print(keys)

When using the python binding's directory function, Databroker hangs silently if the match_template doesn't match any keys.

jmoon1506 commented 4 years ago

Link to python API:

https://github.com/IBM/data-broker/tree/master/bindings/python

Sara-KS commented 4 years ago

Hi @jmoon1506,

What is the behavior if you call the dbr.testKey function with an empty match_template prior to trying the directory function? If the key does not exist, then you can skip the directory call or wait and try again later.

jmoon1506 commented 4 years ago

That seems to work, but in order to use a match_template we check an iterator instead of testKey.

first_key, iterator = dbr.iterator(dbr_hdl, dbr.DBR_ITERATOR_NEW, dbr.DBR_GROUP_EMPTY, match_template)
if first_key != None:
    keys, nKeys, res = dbr.directory(dbr_hdl, match_template, dbr.DBR_GROUP_EMPTY, count, 1000)
lasch commented 4 years ago

@jmoon1506 I think, I found and fixed the issue. Do you need a tagged version in order to test it on your side? Otherwise, I'd wait for you to confirm it's fixed and then would tag this update version.

jmoon1506 commented 3 years ago

Hi @lasch. Sorry for the delay -- I have tested the latest master commit's python bindings, and it works correctly. Please go ahead and tag this updated version.

lasch commented 3 years ago

Thanks @jmoon1506 . New version is tagged as version: 0.7.2