search

ffnord / ffmap-backend

THIS PROJECT DOESN'T HAVE A MAINTAINER!
Other
20 stars 59 forks source link

Prevent inconsistent data in json files. #74

Open tobox opened 8 years ago

tobox commented 8 years ago

Dumping the json content directly into the final output files creates inconsistend data temporarily, which may cause strange effects. Writing to temporary files and atomically renaming them should fix this.

Works only on unix-like systems where os.rename can overwrite existing files.

leahoswald commented 8 years ago

@tobox it would be nice if you fix the problems which let the tests to fail.

jplitza commented 8 years ago

This only fixes a small aspect of the bigger problem that also caused #44, namely that apparently in many instances, the output of this script currently is served directly by webservers. This PR may guarantee an atomic modification to the JSON files, but not to the RRDs or PNGs. So I somehow have the feeling there should be a more general approach to the bigger problem, rather than fixing several small effects it has.

tobox commented 8 years ago

If somebody was working on fixing the bigger problem, I would agree that this PR should not be merged yet. If nobody is working on it, I guess a workaround for real-world problems should be considered for merging, especially since this minor change is trivial to revert.

ecsv commented 8 years ago

I have (unfortunately) seen a complete messed up nodes.json due to some hard reset of a server. I would therefore appreciate such a change. But It would be good when the buffer is actually written to disk before the file is renamed

f.flush()
os.fsync(f.fileno())