search

restic / rest-server

Rest Server is a high performance HTTP server that implements restic's REST backend API.
BSD 2-Clause "Simplified" License
922 stars 138 forks source link

WARNING: fsync is not supported by the data storage #230

Closed r0l1 closed 1 year ago

r0l1 commented 1 year ago

Output of rest-server --version

docker image: f41f6db080c5 Digest: sha256:8b806c911815e78db8a95e7d41f34780d0690eae0d75cf8a49c35c8ec3c831e3

How did you run rest-server exactly?

docker run --name "restic-server" -p 8000:8000 -v /data:/data --env "OPTIONS=--private-repos --append-only" restic/rest-server

What backend/server/service did you use to store the repository?

Expected behavior

fsync working. This is an ext4 drive (raid1 -> luks -> ext4).

Actual behavior

I get hundreds of following messages during a backup process:

WARNING: fsync is not supported by the data storage. This can lead to data loss, if the system crashes or the storage is unexpectedly disconnected.

Steps to reproduce the behavior

Don't know. I compiled a simple go program (via docker golang:alpine) and tested for f.Sync support. No error is thrown and it also runs within the same docker container without any problems.

Do you have any idea what may have caused this?

Maybe the compiled binary within the official docker image differs from my test binary? Different go versions?

Do you have an idea how to solve the issue?

Currently not. The filesystem supports fsync.

Did rest-server help you today? Did it make you happy in any way?

Restic is just awesome and I really appreciate this open source project.

r0l1 commented 1 year ago

And why are there so many log messages? Looking at the code:

fsyncWarning sync.Once

should actually result into one single log message. Any hints?

MichaelEischer commented 1 year ago

Arghh, looks like I've messed up the check for fsync support in #199. https://github.com/restic/rest-server/pull/231 fixes the fsync check. But I'm not yet sure why the warning is printed multiple times.

wojas commented 1 year ago

But I'm not yet sure why the warning is printed multiple times.

A new Handler is created for every request, so it will current be printed at most once per request, which is not very helpful.

Perhaps create a global var for the sync.Once?

MichaelEischer commented 1 year ago

@wojas I've moved the sync.Once instance into the server struct.

DanielGibson commented 1 year ago

Good to see this fixed, I was just gonna report this :)

My setup is similar (harddisk partition -> LUKS encryption -> ext4), and I wasn't 100% sure if the warning maybe is true and due to the LUKS in the middle maybe fsync() on ext4 doesn't go through to the physical disk, but according to https://stackoverflow.com/questions/32225517/fsync-on-mapped-crypted-device-with-dm-crypt it does?

(either way the warning should of course only be logged once, so I'm glad that's fixed)

kittkott commented 1 year ago

Windows Server 2019 rest-server version rest-server 0.12.0 compiled with go1.20.3 on windows/amd64

I get hundreds of following messages during a backup process:

"WARNING: fsync is not supported by the data storage. This can lead to data loss, if the system crashes or the storage is unexpectedly disconnected."

DanielGibson commented 1 year ago

The fix has been implemented on May 18th, but the last release (0.12.0, the one you're using) is from April.

I think a new release would make sense, as those messages are both noisy and (apparently wrongly) alarming. (Says the guy who hasn't done a release of his open source project in over a year 🙃)