search

eladnoor / component-contribution

Standard reaction Gibbs energy estimation for biochemical reactions
Other
17 stars 15 forks source link

Use SQLite for the compound cache instead of CSV #33

Closed eladnoor closed 5 years ago

eladnoor commented 5 years ago

An initial implementation has been made, but we should use SQLite more intelligently as the memory cache itself (i.e. update the DB every time a new compound is cached rather than when dump() is called explicitly).

flamholz commented 5 years ago

pandas works just fine for this purpose as well - it can read csv files and you can query it like a DB. A

On Mon, Nov 19, 2018 at 4:55 AM Elad Noor notifications@github.com wrote:

An initial implementation has been made, but we should use SQLite more intelligently as the memory cache itself (i.e. update the DB every time a new compound is cached rather than when dump() is called explicitly).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/eladnoor/component-contribution/issues/33#issuecomment-439883647, or mute the thread https://github.com/notifications/unsubscribe-auth/ACg2gao5IUoRZBMY6Y8NFWrzgj0Rl601ks5uwqpXgaJpZM4YokgB .

eladnoor commented 5 years ago

In the future, I would like to cache a large amount of compounds (hundreds of thousands perhaps), including their structure, pKas, group vectors, etc. I'm not sure that we will be able to hold all that data in memory at once.

flamholz commented 5 years ago

Got it On Tue, Nov 20, 2018 at 12:16 AM Elad Noor notifications@github.com wrote:

In the future, I would like to cache a large amount of compounds (hundreds of thousands perhaps), including their structure, pKas, group vectors, etc. I'm not sure that we will be able to hold all that data in memory at once.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/eladnoor/component-contribution/issues/33#issuecomment-440182668, or mute the thread https://github.com/notifications/unsubscribe-auth/ACg2gXBJz9ahDXKJTwdwHGm75W-khc_Tks5uw7pogaJpZM4YokgB .