search

typicode / steno

Super fast async file writer with atomic write ⚡
MIT License
678 stars 40 forks source link

Important: works only in a single instance of Node. #10

Closed ciscoheat closed 9 years ago

ciscoheat commented 9 years ago

Hello, does this mean that I cannot run two Node instances that uses steno on the same file simultaneously?

typicode commented 9 years ago

Hi Andreas,

Yes, exactly. It needs to be one Node instance for one file. It's because, it maintains an in-memory lock for each file it writes to.

ciscoheat commented 9 years ago

Ok, that's good to know. I had some strange occurrences when using steno with two different files in the same instance as well. If I did some alternate writing to the files, only one of them saved to disk. When I wrote to only one of them, no problem regardless which one I wrote to. Would this be related?

typicode commented 9 years ago

Not really, it would be a bug. Steno should be able to handle multiple files in the same instance. Do you have a failing code so that I can reproduce?

ciscoheat commented 9 years ago

I'm in the middle of debugging it now actually. It seems like the call to fs.writeFile(tmpFile, data, function(err) { (around line 36) is never completed. Very strange since it worked with another file recently. I will try some more, and see if I can extract a reproducible case.

typicode commented 9 years ago

Awesome thanks :)

ciscoheat commented 9 years ago

I've done some more testing, I was using steno together with a transaction-like scenario (restoring the DB automatically if an operation failed) and probably pushed it too hard, or created some race-condition. When I saved less frequently or used it synchronously it worked, but it made me realize I need to use another database solution. Which made me find OrientDB, so a good thing came out of it at least. :)