search

piskvorky / sqlitedict

Persistent dict, backed by sqlite3 and pickle, multithread-safe.
Apache License 2.0
1.16k stars 130 forks source link

multiple level defaultdict storage #139

Closed careervasu closed 3 years ago

careervasu commented 3 years ago

I have to store a data set of multiple level default dictionary as below:

dataset = defaultdict(lambda: defaultdict(lambda: defaultdict(lambda: defaultdict( lambda: defaultdict( lambda: defaultdict(lambda: defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))))))))

How can i use multiple level nested data structure with sqllitedict? if key is not exists at any level need to create a with a default value without giving a key error. I am fine to use other data structure or any sqllite default dict if supports the functionality.

I am getting issues with pickle the objects and here is the error details:

db = SqliteDict(filename=DB_PATH, tablename=TABLE_JULES_BUILDS, autocommit=True)

pickle.PicklingError: Can't pickle <function at 0x00000000027DBF70>: attribute lookup on main failed

piskvorky commented 3 years ago

Sqlitedict is not a good fit for nested data structures. It's a simple key-value store (backed by SQLite), where "value" is an opaque Python object, treated as a single entity, undifferentiated.