search

emschwartz / ilp3

An implementation of Interledger V3
5 stars 3 forks source link

Multi-process access to DB #13

Open dappelt opened 6 years ago

dappelt commented 6 years ago

Claim writer and channel watcher run in separate processes but access the same db. Did you find a solution for accessing the db from multiple processes? From the levelup docs:

Stores like LevelDB are thread-safe but they are not suitable for accessing with multiple processes. You should only ever have a store open from a single Node.js process. Node.js clusters are made up of multiple processes so a levelup instance cannot be shared between them either. See the aformentioned wiki for modules like multilevel, that may help if you require a single store to be shared across processes.

Did you check multilevel?

emschwartz commented 6 years ago

I didn't find a good solution yet. Right now the XRP plugin is using lowdb, which just writes to a JSON file on disk. I was in the middle of trying to switch it over to using a levelup-compliant store but haven't yet.

Multilevel seems like a good option but is unfortunately out of date and not compatible with the latest levelup version.

I was also looking into whether using RocksDB (Facebook's fork of leveldb) would be an option, because it allows for multiple readers. The Node.js module unfortunately doesn't expose the read-only option right now but it should be possible to add that (see https://github.com/Level/rocksdb/issues/13#issuecomment-346389733)).