search

piskvorky / sqlitedict

Persistent dict, backed by sqlite3 and pickle, multithread-safe.
Apache License 2.0
1.16k stars 131 forks source link

in-memory use #67

Open dineshbvadhia opened 7 years ago

dineshbvadhia commented 7 years ago

A sqlitedict db is created and saved to disk eg.

with SqliteDict('mydb'), flag='c') as db:

db.close() How can you now use 'mydb' in-memory using the sqlite :memory option with sqlitedict?
Digenis commented 7 years ago

@dineshbvadhia, what is your use case?

dineshbvadhia commented 7 years ago

To use the data stored in sqlitedict in-memory for performance reasons. The in-memory db will be read-only.

Digenis commented 7 years ago

What advantages does this offer compared to a regular dict? What do you mean by "read-only db"?

dineshbvadhia commented 7 years ago

Are you saying sqlitedict operates in-memory already just like a regular dict?

Digenis commented 7 years ago

No. I'm asking why would you want to use sqlitedict in memory instead of a plain old dict which is much faster and doesn't even need a package.

dineshbvadhia commented 7 years ago

Will a regular dict support scalability for say 10m to 100m key:value pairs?

Digenis commented 7 years ago

In regards to memory usage, it's not easy to answer but I wouldn't expect improvements. Perhaps you'd want to look into some other technology, like redis.

In regards to speed, a plain dict is least tens of times faster than sqlitedict.

Try benchmarking it. Use as a db filename the :memory: magic string (yes, the sqlite module has a magic string and sqlitedict passes it directly as an argument) This answers your original inquiry – just use SqliteDict(':memory:') @tmylk, @piskvorky, do you consider this a a bug or a feature?