Improve handling of dangling symlinks when creating file list

[prologic]

Hey @stapelberg 👋

Toying around with your tool/library here and ran into a small problem I'm a bit befuddled by: (I'm trying to see if I can build a container backup solution using rsync as the primary driver)

I'm an invoking an rsync(1) client invoking the gokr-rsync over a shell like so:

$ rsync --rsh="docker run -i --rm -v data:/data prologic/docker-rsync" --list-only :
2022/12/30 23:44:31 remote protocol: 31
2022/12/30 23:44:31 exclusion list read
2022/12/30 23:44:31 sendFileList(module="implicit")
2022/12/30 23:44:31   path "." (module root "/")
2022/12/30 23:44:31 lstat /proc/1/fd/3: no such file or directory
gokr-rsync [sender]: lstat /proc/1/fd/3: no such file or directory
rsync: connection unexpectedly closed (71 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(231) [Receiver=3.2.7]

I'm not sure why /proc/1/fd/3 is being opened? I poked around in the container that is spawned and there is no file descriptor 3, only 0 1 2 as you'd expect (stdin, stdout, stderr).

My Dockerfile is mostly similar to yours:

FROM golang:alpine AS build

RUN go install github.com/gokrazy/rsync/cmd/gokr-rsyncd@latest

FROM alpine AS runtime

COPY --from=build /go/bin/gokr-rsyncd /usr/local/bin

#USER nobody:nobody

VOLUME /data

COPY entrypoint.sh /entrypoint

ENTRYPOINT ["/entrypoint"]

And the entrypoint.sh:

#!/bin/sh

cd /data || exit 1
# exec gokr-rsyncd --daemon --gokr.listen=0.0.0.0:8730 --gokr.modulemap=pwd=$PWD
exec gokr-rsyncd --server --sender . .

[prologic]

Just wanted to say that I got something working in a slightly different setup:

Dockerfile:

# Build
FROM golang:alpine AS build

RUN go install github.com/gokrazy/rsync/cmd/gokr-rsyncd@latest

# Runtime
FROM alpine:latest

RUN apk --no-cache -U add su-exec shadow

ENV PUID=1000
ENV PGID=1000

RUN addgroup -g "${PGID}" nonroot && \
    adduser -D -H -G nonroot -h /var/empty -u "${PUID}" nonroot && \
    mkdir -p /data && chown -R nonroot:nonroot /data

EXPOSE 8730

VOLUME /data

WORKDIR /

# force cgo resolver
ENV GODEBUG=netdns=cgo

COPY --from=build /go/bin/gokr-rsyncd /usr/local/bin/rsyncd

COPY entrypoint.sh /init

ENTRYPOINT ["/init"]
CMD ["rsyncd", "--daemon", "--gokr.listen=0.0.0.0:8730", "--gokr.modulemap=data=/data"]

entrypoint.sh:

#!/bin/sh

[ -n "${PUID}" ] && usermod -u "${PUID}" nonroot
[ -n "${PGID}" ] && groupmod -g "${PGID}" nonroot

printf "Switching UID=%s and GID=%s\n" "${PUID}" "${PGID}"
exec su-exec nonroot:nonroot "$@"

I was successfully able to rsync the contents of a Docker named volume (even while a container was running/attached to it) 👌 -- That's backup done.

Now for restore 😅

[prologic]

My plan was to (btw):

• Spin up a backup container like: docker run --rm -p 8730:8730 -v data:/data:ro prologic/docker-rsync for any given named volume.
• Perform an initial backup using like: rsync --archive --delete --verbose --port 8730 rsync://localhost/data/ .
• Temporarily stop the any attached running containers.
• Re-run the backup (in case any files changed)
• Restart the affected containers.

[stapelberg]

Hey! The problem you ran into is that you were trying to serve /, which includes the /proc pseudo file system, which in turn contains a bunch of dangling symlinks (by design).

Obviously the current behavior of gokr-rsyncd isn’t great when encountering dangling symlinks. I haven’t checked what the original rsync does yet.

You can work around the issue by changing the code to ignore ENOENT for now:

--- i/rsyncd/flist.go
+++ w/rsyncd/flist.go
@@ -45,6 +45,12 @@ func (st *sendTransfer) sendFileList(mod Module, opts *Opts, paths []string) (*f
        err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
            // st.logger.Printf("filepath.WalkFn(path=%s)", path)
            if err != nil {
+               if os.IsNotExist(err) {
+                   // We encounter -ENOENT when walking over a dangling symlink
+                   // (e.g. /proc/<pid>/fd/3). Don’t stop file list creation,
+                   // just ignore the affected file.
+                   return nil
+               }
                return err
            }

[prologic]

@stapelberg Oh! 🤦‍♂️ This is because despite the modulemap I configured, I had set the working directory of the image to / 🤔