Container does not start when /data is NFS mount without no_root_squash

vassilmladenov commented 3 years ago

Is there a pinned issue for this?

[X] I have read the pinned issues and could not find my issue

Is there an existing or similar issue/discussion for this?

[X] I have searched the existing issues
[X] I have searched the existing discussions

Is there any comment in the documentation for this?

[X] I have read the documentation, especially the FAQ and Troubleshooting parts

Is this related to a provider?

[X] I have checked the provider repo for issues
[X] My issue is NOT related to a provider

Are you using the latest release?

[X] I am using the latest release

Have you tried using the dev branch latest?

[X] I have tried using dev branch

Config used

docker-compose.yml

  transmission-openvpn:
    cap_add:
      - NET_ADMIN
    volumes:
      - /mnt/bittorrent:/data
    environment:
      - PUID=1000
      - PGID=1000
      - OPENVPN_PROVIDER=SURFSHARK
      - OPENVPN_CONFIG=xxxxx
      - OPENVPN_USERNAME=xxxxx
      - OPENVPN_PASSWORD=xxxxx
      - LOCAL_NETWORK=xxxxx
      - TRANSMISSION_WEB_UI=flood-for-transmission
    logging:
      driver: json-file
      options:
        max-size: 10m
    ports:
      - 9091:9091
    image: haugene/transmission-openvpn

Current Behavior

/mnt/bittorrent is an NFS mount. The mount is configured without no_root_squash on the NFS server.

I get this error when I start the program

transmission-openvpn_1  | Traceback (most recent call last):
transmission-openvpn_1  |   File "/etc/transmission/updateSettings.py", line 90, in <module>
transmission-openvpn_1  |     with open(transmission_settings, 'w') as fp:
transmission-openvpn_1  | PermissionError: [Errno 13] Permission denied: '/data/transmission-home/settings.json'

On the client, mkdir /mnt/transmission/transmission-home works but sudo mkdir /mnt/transmission/transmission-home does not (as expected). I expected the PUID/PGID variables to make the container with the correct permissions.

I confirmed uid=1000(vmladenov) gid=1000(vmladenov) on the client with id -a.

Expected Behavior

It creates a directory at /mnt/bittorrent/transmission-home and runs correctly

How have you tried to solve the problem?

I can make this work by making the NFS mount no_root_squash, but then all of the directories have root:root permissions instead of vmladenov:vmladenov.

Log output

vmladenov@aurora:~$ docker-compose up
Starting vmladenov_transmission-openvpn_1 ... done
Attaching to vmladenov_transmission-openvpn_1
transmission-openvpn_1  | Starting container with revision: 6ee67f8210d7f9ca17677c2b108390acc91ec5ab
transmission-openvpn_1  | Creating TUN device /dev/net/tun
transmission-openvpn_1  | Using OpenVPN provider: SURFSHARK
transmission-openvpn_1  | Running with VPN_CONFIG_SOURCE auto
transmission-openvpn_1  | No bundled config script found for SURFSHARK. Defaulting to external config
transmission-openvpn_1  | Downloading configs from https://github.com/haugene/vpn-configs-contrib/archive/main.zip into /tmp/tmp.kSW5SUMstc
transmission-openvpn_1  | Extracting configs to /tmp/tmp.qGvyD1p5aI
transmission-openvpn_1  | Found configs for SURFSHARK in /tmp/tmp.qGvyD1p5aI/vpn-configs-contrib-main/openvpn/surfshark, will replace current content in /etc/openvpn/surfshark
transmission-openvpn_1  | Cleanup: deleting /tmp/tmp.kSW5SUMstc and /tmp/tmp.qGvyD1p5aI
transmission-openvpn_1  | Starting OpenVPN using xxxxx
transmission-openvpn_1  | Modifying /etc/openvpn/surfshark/xxxxx.ovpn for best behaviour in this container
transmission-openvpn_1  | Modification: Point auth-user-pass option to the username/password file
transmission-openvpn_1  | Modification: Change ca certificate path
transmission-openvpn_1  | Modification: Change ping options
transmission-openvpn_1  | Setting OpenVPN credentials...
transmission-openvpn_1  | adding route to local network xxx.xxx.xxx.xxx/24 via 172.20.0.1 dev eth0
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 19 2021
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 library versions: OpenSSL 1.1.1f  31 Mar 2020, LZO 2.10
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 NOTE: --fast-io is disabled since we are not using UDP
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 TCP/UDP: Preserving recently used remote address: [AF_INET]185.124.240.145:1443
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 Socket Buffers: R=[131072->131072] S=[16384->16384]
transmission-openvpn_1  | Sat Sep 18 08:01:37 2021 Attempting to establish TCP connection with [AF_INET]185.124.240.145:1443 [nonblock]
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 TCP connection established with [AF_INET]185.124.240.145:1443
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 TCP_CLIENT link local: (not bound)
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 TCP_CLIENT link remote: [AF_INET]185.124.240.145:1443
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 TLS: Initial packet from [AF_INET]185.124.240.145:1443, sid=f048619d 7b307924
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 VERIFY OK: depth=2, C=VG, O=Surfshark, CN=Surfshark Root CA
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 VERIFY OK: depth=1, C=VG, O=Surfshark, CN=Surfshark Intermediate CA
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 VERIFY KU OK
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 Validating certificate extended key usage
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 VERIFY EKU OK
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 VERIFY OK: depth=0, CN=xxxxx.prod.surfshark.com
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1635', remote='link-mtu 1583'
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM'
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 Control Channel: TLSv1.2, cipher TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
transmission-openvpn_1  | Sat Sep 18 08:01:38 2021 [xxxxx.prod.surfshark.com] Peer Connection Initiated with [AF_INET]185.124.240.145:1443
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 SENT CONTROL [xxxxx.prod.surfshark.com]: 'PUSH_REQUEST' (status=1)
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 162.252.172.57,dhcp-option DNS 149.154.159.92,redirect-gateway def1,sndbuf 524288,rcvbuf 524288,explicit-exit-notify,block-outside-dns,route-gateway 10.7.7.1,topology subnet,ping 60,ping-restart 180,ifconfig 10.7.7.2 255.255.255.0,peer-id 0,cipher AES-256-GCM'
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 Options error: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:7: block-outside-dns (2.4.7)
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: timers and/or timeouts modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: --explicit-exit-notify can only be used with --proto udp
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: --sndbuf/--rcvbuf options modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 Socket Buffers: R=[131072->425984] S=[87040->425984]
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: --ifconfig/up options modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: route options modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: route-related options modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: peer-id set
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: adjusting link_mtu to 1658
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 OPTIONS IMPORT: data channel crypto options modified
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 Data Channel: using negotiated cipher 'AES-256-GCM'
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 ROUTE_GATEWAY 172.20.0.1/255.255.0.0 IFACE=eth0 HWADDR=02:42:ac:14:00:02
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 TUN/TAP device tun0 opened
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 TUN/TAP TX queue length set to 100
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 /sbin/ip link set dev tun0 up mtu 1500
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 /sbin/ip addr add dev tun0 10.7.7.2/24 broadcast 10.7.7.255
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 /etc/openvpn/tunnelUp.sh tun0 1500 1586 10.7.7.2 255.255.255.0 init
transmission-openvpn_1  | Up script executed with tun0 1500 1586 10.7.7.2 255.255.255.0 init
transmission-openvpn_1  | Updating TRANSMISSION_BIND_ADDRESS_IPV4 to the ip of tun0 : 10.7.7.2
transmission-openvpn_1  | Using Flood for Transmission UI, overriding TRANSMISSION_WEB_HOME
transmission-openvpn_1  | Updating Transmission settings.json with values from env variables
transmission-openvpn_1  | mkdir: cannot create directory '/data/transmission-home': Permission denied
transmission-openvpn_1  | Generating settings.json for Transmission from environment and defaults /etc/transmission/default-settings.json
transmission-openvpn_1  | Overriding bind-address-ipv4 because TRANSMISSION_BIND_ADDRESS_IPV4 is set to 10.7.7.2
transmission-openvpn_1  | Overriding download-dir because TRANSMISSION_DOWNLOAD_DIR is set to /data/completed
transmission-openvpn_1  | Overriding incomplete-dir because TRANSMISSION_INCOMPLETE_DIR is set to /data/incomplete
transmission-openvpn_1  | Overriding rpc-port because TRANSMISSION_RPC_PORT is set to 9091
transmission-openvpn_1  | Overriding watch-dir because TRANSMISSION_WATCH_DIR is set to /data/watch
transmission-openvpn_1  | Traceback (most recent call last):
transmission-openvpn_1  |   File "/etc/transmission/updateSettings.py", line 90, in <module>
transmission-openvpn_1  |     with open(transmission_settings, 'w') as fp:
transmission-openvpn_1  | FileNotFoundError: [Errno 2] No such file or directory: '/data/transmission-home/settings.json'
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 /sbin/ip route add 185.124.240.145/32 via 172.20.0.1
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 /sbin/ip route add 0.0.0.0/1 via 10.7.7.1
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 /sbin/ip route add 128.0.0.0/1 via 10.7.7.1
transmission-openvpn_1  | Sat Sep 18 08:01:39 2021 Initialization Sequence Completed

Environment

- OS: Debian 10
- Docker: version 20.10.6, build 370c289

Anything else?

No response

pkishino commented 3 years ago

@haugene any idea ? @vassilmladenov when you check the id, are you checking this inside the docker ? this looks like some issue with linux nfs mount and docker and not so much a bug with this container..should consider moving this to discussions instead

vassilmladenov commented 3 years ago

@pkishino I’m checking the id in the host. The reason I made an issue here is that until a few days ago I was using linuxserver/transmission on this very mount with PUID/PGID set to 1000 and it was working fine.

vassilmladenov commented 3 years ago

I looked at this again and I think the issue is the user that runs the setup script of the container, not the PGID. I did the following:

set the mount to root_squash
made a transmission-home in my home directory
added an entry in volumes - /home/vmladenov/transmission-home:/config/transmission-home
set the environment variable TRANSMISSION_HOME=/config/transmission-home

I used /config because I needed something outside /data and I saw the container sets up a /config volume.

On the next run, it got past the setup and the container started properly. Plus, the files get the correct permissions. I've updated the title of the issue. It seems to be the same problem described in #1828

pkishino commented 3 years ago

@haugene Can we get the scripts to simply be run by PGID if this is set?

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

stale[bot] commented 2 years ago

Feel free to re-open this issue if you think it deserves another look.

ndanyluk commented 2 years ago

Hi just wanted to add that I'm seeing this as well and so far not sure how to get around it. Setting the container to privileged mode solved it for me, but I don't want to give the container that much capability if only NET_ADMIN should be required. I have my NFS export set up with root_squash and anonuid=1000, anongid=1000 so I should be able to get things running, but I am stuck in a crash loop

fergbrain commented 2 years ago

I can confirm this is also an issue. I worked around it by attaching to the docker container, creating a new user, run python3 /etc/transmission/updateSettings.py /etc/transmission/default-settings.json ${TRANSMISSION_HOME}/settings.json from /etc/transmission/start.sh, then comment out that line, restart the container and then it worked...obviously not ideal.

Update! ...setting TRANSMISSION_HOME=/etc/transmission/. The downside is that your transmission settings won't be persistent. It also looks like you'll lose the actual .torrent files too.

sudopseudocode commented 2 years ago

I am posting here because I came across this issue as well and would like to share my solution similar to what was proposed above.

Set TRANSMISSION_HOME=/transmission-home env var. And I map this to a local folder on the docker host -v /home/your-user/transmission:/transmission-home. This solves any permission issues with the files handled within that folder. And I keep my /data directory mapped to a folder on my NFS share, which is where everything is downloaded anyways (/data/completed & /data/incompleted).

Some additional info about my set up, in case it might affect anything:

Docker host is running Ubuntu on a VM in Proxmox.
NFS share is mounted in /etc/fstab with these settings: defaults,_netdev
Used OMV to configure NFS share, created a matching user under the ACL section which has matching UID/GID (1000)
Using PUID=1000 & PGID=1000, which matches my docker user.

haugene / docker-transmission-openvpn