DataDog / kvexpress

## Auto-archived due to inactivity. ## Go program to move data in and out of Consul's KV store.
Apache License 2.0
128 stars 13 forks source link

Writing to a file should be atomic #88

Closed calebdoxsey closed 8 years ago

calebdoxsey commented 8 years ago

Currently it's possible for downstream readers to read a partially written file because WriteFile is not atomic. Ideally an updated file would only change once, and the change would be total, so that you'd either read the old state or the new state, but nothing in between.

One way to implement this is by writing to a temporary file and then renaming the file:

func safeWriteFile(dst string, data []byte) error {
    tmp := dst + ".tmp"
    defer os.RemoveAll(tmp)

    err := ioutil.WriteFile(tmp, data, 0755)
    if err != nil {
        return err
    }
    return os.Rename(tmp, dst)
}

This works because renaming a file in unix is atomic.

One thing to keep in mind that is that it's only atomic on the same physical device. So writing to /tmp may not work.

darron commented 8 years ago

@Arkelenia and I were just talking about to this earlier today.

Am looking at getting a patch in shortly.

darron commented 8 years ago

Right now ioutil.WriteFile truncates the file first - so that's likely what you're seeing.

Our main use has been with dnsmasq which doesn't read the file until you send a SIGHUP so we haven't been hitting this.

darron commented 8 years ago

Fixed. Will test in staging.