fabiospampinato / atomically

Read and write files atomically and reliably.
MIT License
161 stars 9 forks source link

Explore ignoring intermediate writes #2

Open fabiospampinato opened 4 years ago

fabiospampinato commented 4 years ago

If a write has been initiated, and another 10 writes to the same path have got queued up in the meantime, it's possible that the computer will crash before finishing all those writes, maybe it would be better to just skip intermediate writes and go straight to the last write queued up.

aleksey-hoffman commented 3 years ago

I agree, there's no point writing intermediate data. If the program requested to write some value 100 times in 1 second (which can happen when you use a slider to modify some setting quickly), there's no point writing all those changes, just write the last one. Though, it would probably be better to make this approach optional.

I think the current approach is also the cause of another problem. When I do exactly what I described above (modifying a value multiple times in a row quickly), I get the following error:

Uncaught (in promise) Error: Error: Error: Error: EPERM: operation not permitted, 
rename 'C:\app_path\settings.json.tmp-6623291918d97ceb' -> 'C:\app_path\settings.json'

I fixed it by wrapping this module's writeFile function with a custom throttle class instance that reduces the amount of calls to the function (calls once every 250 ms) and schedules the incoming values while waiting between the calls, but it doesn't create a queue, it just overwrites the existing values and then writes them when the 250ms waiting period is over.

Example:

// 1. Request a lot of consecutive writes
write({ key: 'imagePositionX', value: '12'})
write({ key: 'imagePositionX', value: '14'})
write({ key: 'imagePositionX', value: '15'})
write({ key: 'imagePositionY', value: '2'})

// 2. 
// - If throttle is waiting, schedule properties (overwrite with incoming values)
// - If throttle is running, write the properties from 'scheduledData'
const scheduledData = [
  { key: 'imagePositionX', value: '15'},
  { key: 'imagePositionY', value: '2'}
]
fabiospampinato commented 3 years ago

Triggering hundreds of writes to the same file in quick succession is kind of a separate problem at the app level, you probably shouldn't do that to being with.