how about replace pickle with dill? or at least make it an option?

[mo-han]

Queue object couldn't be pickled by pickle, which would raise:

TypeError: can't pickle _thread.lock objects

luckily, dill is API-compatible with pickle, and it could pickle (almost) anything in python, including Queue

so, could you please add an option for that?

[piskvorky]

Serializing locks makes no sense. What is your application? This looks more like a programmer error, in which case changing serialization formats wouldn't help.

[mo-han]

Serializing locks makes no sense

https://stackoverflow.com/a/25411691/7966259

[piskvorky]

I know what dill is, but I'm unclear why you need it. Can you describe your motivation?

Attempting to serialize objects that are fundamentally unserializable (file handles, open streams, generators, locks…) might indicate an error in the application layer, in which case masking the problem at the database library layer by replacing pickle with dill would do more harm than good, even if it superficially "works".

[mo-han]

@piskvorky that stackoverflow answer is not to tell you what dill is, but to show there are some people who need to pickle those so called "un-serializable" objects, and which is practicable. didn't see anyone asking for a motivation in that stackoverflow answer. why would you like to know my motivation so much, when you already had that motivation from others? even without your help, the patch is quite simple:

import dill
import sqlitedict

sqlitedict.dumps = dill.dumps
sqlitedict.loads = dill.loads
sqlitedict.PICKLE_PROTOCOL = dill.HIGHEST_PROTOCOL

no big deal have a nice day

[piskvorky]

I didn't know what your comment-less link was supposed to show me.

We do not include patches without a clear motivation. Especially ones with dangerous side effects.

If dill works for you, great – but in case anyone stumbles upon this ticket in the future, please mind the caveats above.

[mo-han]

def sqlitedict_with_dill():
    import dill
    import sqlitedict
    sqlitedict.dumps = dill.dumps
    sqlitedict.loads = dill.loads
    sqlitedict.PICKLE_PROTOCOL = dill.HIGHEST_PROTOCOL
    return sqlitedict

import queue

sqlitedict = sqlitedict_with_dill()
q = queue.Queue()
q.put(1)
with sqlitedict.SqliteDict('tmp.db', autocommit=True) as sd:
    sd['q'] = q
with sqlitedict.SqliteDict('tmp.db', autocommit=True) as sd:
    print(sd['q'].get() == 1)