search

peterbourgon / diskv

A disk-backed key-value store.
http://godoc.org/github.com/peterbourgon/diskv
MIT License
1.41k stars 102 forks source link

Implement atomic write to disk #34

Closed apelisse closed 7 years ago

apelisse commented 7 years ago

Implement atomic write by writing into a temporary file (within the same directory to guarantee that it won't move accross disks), with the same permission and then rename at the last moment to make it available.

apelisse commented 7 years ago

This is specifically important when used with httpcache, as we want to be sure that we don't have partial writes to disk, or even concurrent read/write.

peterbourgon commented 7 years ago

Nice! Can you come up with a test that would succeed with atomic ops and fail without them?

apelisse commented 7 years ago

Yeah. We need to come-up with a test that breaks specifically during the Copy. That should be easy to do if we use a sort of BrokenReader (I'll see if something already exists to do that) as the input of WriteStream. Working on this now.

apelisse commented 7 years ago

New test is here! I've tested with and without the initial patch, and I'm more confident that it works now :-)

peterbourgon commented 7 years ago

Badass. Thanks!