opencybersecurityalliance / firepit

Firepit - STIX Columnar Storage
Apache License 2.0
15 stars 12 forks source link

asyncstorage tries to use sync functions from sqlstorage #73

Closed pcoccoli closed 1 year ago

pcoccoli commented 1 year ago

In AsyncStorage.lookup():

            if not col_dict:
                col_dict = _get_col_dict(self)

But _get_col_dict comes from sqlstorage.py and is not async:

def _get_col_dict(store):
    q = Query('__columns')
    col_dict = defaultdict(list)
    results = store.run_query(q).fetchall()
    for result in results:
        col_dict[result['otype']].append(result['path'])
    return col_dict

Results in a backtrace with "AttributeError: 'coroutine' object has no attribute 'fetchall'"

A workaround is to pass in a col_dict to lookup and avoid the bug. To generate col_dict (if you haven't already):

dbcache = await get_dbcache(store)
col_dict = dbcache.col_dict

This is probably more efficient anyway.

The real fix is to re-implement an async version of _get_col_dict in asyncstorage.py.

pcoccoli commented 1 year ago

This has been fixed, but I can't remember which commit it was.