Performance: Pickle an Object each value change ?

[instasck]

I want to have a dict contain objects:

class MyClass:
  x = 0
  y = 0

Will it pickle the whole object and write it to disk (on autocommit) every time only x is changing ? Can we do anything to only modify x on disk ?

[jonchun]

https://github.com/RaRe-Technologies/sqlitedict/blob/master/sqlitedict.py#L248

It calls self.encode(value), and value would be your object in this case, so yes, the entire object would be pickled again and written to disk on autocommit.

How large are your objects? Are you running into performance issues, or is this a theoretical problem?

[piskvorky]

The question seems confused. Sqlitedict doesn't track whether "x is changing". It will simply serialize objects to disk (using pickle), and allow their retrieval via a key using the SQLite database.

Whether x is a class attribute (as in your example) or an instance attribute, or an entire insitance, or some other variable, is of no consequence. They're all atomic objects, from sqlitedict's perspective.

[sionking]

The question seems confused. Sqlitedict doesn't track whether "x is changing". It will simply serialize objects to disk (using pickle), and allow their retrieval via a key using the SQLite database.

Whether x is a class attribute (as in your example) or an instance attribute, or an entire insitance, or some other variable, is of no consequence. They're all atomic objects, from sqlitedict's perspective.

well I guess pickling just an attribute and not the entire object can have performance impackt.

[piskvorky]

Of course – store just the mutating attribute, if you have a large object that otherwise mutates infrequently and pickles slowly. The stored keys and values are always atomic.