rclone mount problems on docker container

[fawzibr]

What is the problem you are having with rclone?

I'm using rclone mount in a docker container and I have been pulling my hair because I thought I was configuring something wrong. I was installing the latest version (using the script) and it was not mounting. Created a small docker container that just 'rclone lsf' and ' rclone mount + ls' and instead of using the latest version used the debian packaged one (v1.60.1-DEV) and it worked. Used the latest one and it fails. So something changed between 1.60 and 1.64 that broke mounting from docker for me.

What is your rclone version (output from rclone version)

rclone v1.64.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.2.0-32-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: none

Previous version failed as well (1.63.1)

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

ubuntu 22.04 (64 bit) docker version: Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1 docker image: bitnami/minideb:bookworm

Which cloud storage system are you using? (e.g. Google Drive)

Testing with smb, but dont think backend matters since it 'rclone lsf'

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

Command:

docker run --rm --device /dev/fuse --privileged -e PROTOCOL=smb -e HOST=vm-sql2008 -e LOCATION=/Backup/ -e OPTIONS="user=my_user,pass=$(rclone obscure my_password)" test-mount

Dockerfile:

FROM bitnami/minideb:bookworm
# add non-free repository
RUN sed -i -e's/ main/ main contrib non-free/g' /etc/apt/sources.list
# install packages
RUN apt-get update
RUN install_packages fuse
RUN install_packages curl
RUN install_packages gpg
RUN install_packages ca-certificates
RUN install_packages unzip
# debian version
# RUN install_packages rclone
# rclone.org version
RUN curl https://rclone.org/install.sh | bash
# copy files
COPY test.sh test.sh
ENTRYPOINT [ "./test.sh" ]

test.sh:

#!/bin/bash

echo "DOCKER MOUNT TEST"
if [ -z "$PROTOCOL" ]; then
    echo "  Environment variable PROTOCOL not set"
    exit    
elif [ -z "$HOST" ]; then
    echo "  Environment variable HOST not set"
    exit    
elif [ -z "$LOCATION" ]; then
    echo "  Environment variable LOCATION not set"
    exit
fi

ARGS=":$PROTOCOL,host=$HOST"
if [ -n "$OPTIONS" ]; then
    ARGS="$ARGS,$OPTIONS"
fi

ARGS="$ARGS:$LOCATION"

echo "--------------------"
echo "TEST LSF"

rclone lsf $ARGS -vv --max-depth=1

echo "--------------------"
echo "TEST MOUNT"

mkdir /tmp/src

rclone mount --daemon --allow-other --allow-non-empty --vfs-cache-mode writes $ARGS /tmp/src -vv

ls /tmp/src/

umount /tmp/src

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

Log using latest version from rclone.org

DOCKER MOUNT TEST
--------------------
TEST LSF
2023/09/12 14:14:52 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "lsf" ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" "-vv" "--max-depth=1"]
2023/09/12 14:14:52 DEBUG : Creating backend with remote ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/"
2023/09/12 14:14:52 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2023/09/12 14:14:52 DEBUG : :smb: detected overridden config - adding "{9Rx5j}" suffix to name
2023/09/12 14:14:52 DEBUG : fs cache: renaming cache item ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" to be canonical ":smb{9Rx5j}:Backup"
Corefig-main/
Test.txt
2023/09/12 14:14:52 DEBUG : 7 go routines active
2023/09/12 14:14:52 DEBUG : smb://my_user@vm-sql2008:445/Backup/: Closing 1 unused connections
--------------------
TEST MOUNT
2023/09/12 14:14:52 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "mount" "--daemon" "--allow-other" "--allow-non-empty" "--vfs-cache-mode" "writes" ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" "/tmp/src" "-vv"]
2023/09/12 14:14:52 DEBUG : Creating backend with remote ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/"
2023/09/12 14:14:52 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2023/09/12 14:14:52 DEBUG : :smb: detected overridden config - adding "{9Rx5j}" suffix to name
2023/09/12 14:14:52 DEBUG : fs cache: renaming cache item ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" to be canonical ":smb{9Rx5j}:Backup"
2023/09/12 14:15:52 DEBUG : Daemon timed out. Terminating daemon pid 21
2023/09/12 14:15:52 Fatal error: mount not ready: /tmp/src
umount: /tmp/src: not mounted.

Log using version packaged with debian

DOCKER MOUNT TEST
--------------------
TEST LSF
2023/09/12 14:24:30 DEBUG : rclone: Version "v1.60.1-DEV" starting with parameters ["rclone" "lsf" ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" "-vv" "--max-depth=1"]
2023/09/12 14:24:30 DEBUG : Creating backend with remote ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/"
2023/09/12 14:24:30 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2023/09/12 14:24:30 DEBUG : :smb: detected overridden config - adding "{G6lK7}" suffix to name
2023/09/12 14:24:30 DEBUG : fs cache: renaming cache item ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" to be canonical ":smb{G6lK7}:Backup"
2023/09/12 14:24:30 DEBUG : 5 go routines active
2023/09/12 14:24:30 DEBUG : smb://my_user@vm-sql2008:445/Backup/: Closing 1 unused connections
Corefig-main/
Test.txt
--------------------
TEST MOUNT
2023/09/12 14:24:31 DEBUG : rclone: Version "v1.60.1-DEV" starting with parameters ["rclone" "mount" "--daemon" "--allow-other" "--allow-non-empty" "--vfs-cache-mode" "writes" ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" "/tmp/src" "-vv"]
2023/09/12 14:24:31 DEBUG : Creating backend with remote ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/"
2023/09/12 14:24:31 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2023/09/12 14:24:31 DEBUG : :smb: detected overridden config - adding "{G6lK7}" suffix to name
2023/09/12 14:24:31 DEBUG : fs cache: renaming cache item ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" to be canonical ":smb{G6lK7}:Backup"
2023/09/12 14:24:31 DEBUG : rclone: Version "v1.60.1-DEV" finishing with parameters ["/usr/bin/rclone" "mount" "--daemon" "--allow-other" "--allow-non-empty" "--vfs-cache-mode" "writes" ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" "/tmp/src" "-vv"]
Corefig-main
Test.txt

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

[ncw]

2023/09/12 14:14:52 DEBUG : fs cache: renaming cache item ":smb,host=vm-sql2008,user=my_user,pass=my_password:/Backup/" to be canonical ":smb{9Rx5j}:Backup" 2023/09/12 14:15:52 DEBUG : Daemon timed out. Terminating daemon pid 21

The mount took more than one minute to start up so it was killed.

That is the thing to investigate.

Try just doing rclone lsf not in a docker container and see if that works. If it does try the docker container.

This kind of long delay is usually some network timeout.

[fawzibr]

Look at the logs again.

If you see the logs I posted is the output of the docker container, the 'TEST LSF' is 'rclone lsf' and it works inside the container. The 'TEST MOUNT' is 'rclone mount' with the same parameters and it fails with version 1.64.

The second log is the same I just used rclone packaged with debian (v1.60) where 'rclone lsf' and 'rclone mount' work.

So 'rclone lsf' works with v1.60 and v1.64, 'rclone mount' works with v1.60 but not v1.64, how can it be a network timeout?

BTW, both computers are next to each other, on a gigabit network.

[ncw]

Can you try removing the --daemon flag? I suspect that is at the root of the issue.

Also can you reproduce this not in a docker container?

[fawzibr]

Ok, found the issue.

Removed the --daemon flag, it gave me a 'fusermount3 not found'. Package fuse was installed but there is a fuse3 package. Installed fuse3 package and ran without --daemon, it hanged. Ran with --daemon it worked. Tested 1.60 with fuse3 and it worked as well.

So between 1.60 and 1.64 something in fuse changed where it hanged in 1.64 and now it needs fuse3.

Anyway all is well now. Thanks for the help.

[ncw]

Glad you fixed it. Yes we needed to upgrade to fuse3.

I don't recommend --daemon if you can avoid it for reasons just like this!

[fawzibr]

Isn't --daemon required for it to run on the background so I can run other commands on said mount? How should I do something like

rclone mount "remote_connection_string" /mount/remote --daemon
cat /mount/remote/my_file.txt

You mean instead of --daemon I should use:

rclone mount "remote_connection_string" /mount/remote &

Or is there some other way to do it?

[ncw]

I usually recommend systemd for managing mounts.

That seems like overkill for your case though. Carry on using --daemon or use & either should work, just be aware that when --daemon goes wrong it can be difficult to debug!