Overwrite leads to non-deterministic behaviour

[sleepyfox]

The README says:

steno('file.txt').write('C') // still writing A, B is replaced by C

Why on earth would I want C to overwrite B? I expect all f.write operations to queue and complete in order, not for stuff to be silently dropped with no errors! This gives non-deterministic behaviour, e.g.

var file_handle = steno('file.txt'); file_handle.write(A) var B = do_something(); file_handle.write(B); var C = do_something_else(); file_handle.write(C);

I now have no idea whether my file looks like "ABC" or "AB" because this depends on the completion times of .write(B) and do_something_else()...

[typicode]

Hi @sleepyfox

I think there's 2 subjects.

Why C overwrites B ?

Let's say, there was a queue:

var q = ['B', 'C', 'D', 'E', 'F']

Basically, steno takes a shortcut since writing mode is writing and not append to make things faster. The idea is if I know that my file must contain at the end F let's only write that and not the intermediate data.

Non-deterministic behaviour

To have a deterministic behavior, I don't know other ways than:

fs.writeFileSync('file.txt', A)
var B = doSomething()
fs.writeFileSync('file.txt', B)
var C = doSomethingElse()
fs.writeFileSync('file.txt', C)

Or

fs.writeFile('file.txt', A, function(err) {
  var B = doSomething()
  fs.writeFile('file.txt', B, function(err) {
    var C = doSomethingElse()
    fs.writeFileSync('file.txt', C)
  }
}

If you happen to write a lot to file.txt, it will be deterministic but slower.

So, yes steno is a compromise. You tell it to write data and kind of forget about it. The main point of steno is that at the end, you won't have a race condition and that it goes fast.

You can get some kind of control/notification though through callbacks.

For example, you could set a current attribute though that will hold last written data.

var writer = steno('file.txt').setCallback(function(err, data, next) {
  this.current = data
  next()
})

writer.write('A')
// ..
writer.current // A

Just curious, what's your use case and why do you need it to be deterministic?

[sleepyfox]

My current use case is "write a bunch of output to a file", where that output is scores calculated by retrieving data from web APIs. I need to make sure that all contents are written, but (in this case) I don't care too much what order they are in. What I need to make sure is that ALL data is written, and that none is missed.

I really can't imagine ANY situation where I'm writing data to a file and I don't care if something gets silently dropped. I'd be really interested to see any use case where this is OK.

[typicode]

steno is a module extracted from https://github.com/typicode/lowdb

My use case was: I have an object that is regularly modified and I want to persist the whole object often and asynchronously.

So just using fs.writeFile() was not enough.

function write(obj) {
  fs.writeFile('file.txt', JSON.stringify(obj), function() {}
}

var obj = {}
obj.a = 1
write(obj)
obj.a = 2
write(obj)

// file.json would contain {a:1} or {a:2}

But considering your use case, I wouldn't use steno for it either since data is appended.

[sleepyfox]

Ah, mutable state. Enough said.