Certain Transmission settings.json values can only be set by ENV var

phipolis commented 4 months ago

Is there a pinned issue for this?

[X] I have read the pinned issues and could not find my issue

Is there an existing or similar issue/discussion for this?

[X] I have searched the existing issues
[X] I have searched the existing discussions

Is there any comment in the documentation for this?

[X] I have read the documentation, especially the FAQ and Troubleshooting parts

Is this related to a provider?

[X] I have checked the provider repo for issues
[X] My issue is NOT related to a provider

Are you using the latest release?

[X] I am using the latest release

Have you tried using the dev branch latest?

[ ] I have tried using dev branch

Docker run config used

services:
 transmission:
  image: haugene/transmission-openvpn:latest
  cap_add:
    - NET_ADMIN
  devices:
    - /dev/net/tun
  dns:
    - 1.1.1.1
  sysctls:
    - net.ipv6.conf.all.disable_ipv6=0
  restart: always
  ports:
    - "127.0.0.1:9092:9091"
  volumes:
    - /etc/localtime:/etc/localtime:ro 
    - /path/transmission-openvpn:/config
    - /path/Torrents:/data
    ## Previous config:
    # - /Path/Torrents:/data/completed 
    # - /Path/Torrents/incomplete:/data/incomplete 
    # - /Path/Torrents/transmission-home:/config/transmission-home/
  logging:
    options:
      max-size: 10m
  environment:
    - TZ=[redacted]
    - OPENVPN_PROVIDER=[redacted] 
    - OPENVPN_USERNAME=[redacted]
    - OPENVPN_PASSWORD=m
    - CREATE_TUN_DEVICE=false
    - OPENVPN_CONFIG=[redacted]
    - OPENVPN_OPTS=--inactive 3600 --ping 10 --ping-exit 60
    - LOCAL_NETWORK=192.168.1.0/24
    - WEBPROXY_ENABLED=false
    - PGID=100
    - PUID=1026

Current Behavior

I had gotten my setup working in the pre-4.2 days by bind-mounting onto the paths /data/completed, /data/incomplete, and /config/transmission-home. Urgent warnings in the logs prompted me to change my mounts and benefit from the new persistent config. When I did so, some options in the config file inexplicably got overwritten by ENV vars I hadn't set (by the project Dockerfile and updateSettings.py).

Expected Behavior

I expected ALL the transmission values to be usable from settings.json. When I realized there were exceptions that must go through ENV, expected they would have been listed in the docs. That the file described as persistent and safe to edit was getting overwritten with hidden defaults was not expected.

(As an aside, I also think that reading docker secrets and writing them out to files on a bind-mounted volume is pretty surprising behavior and not what most people think is going to happen when they use them. One might go as far as saying it defeats secrets' purpose.)

How have you tried to solve the problem?

Much reading of source and triple checking my config until I understood what was happening

Log output

Starting container with revision: 07f5a2b9aea5028c9bb75438c1552708e91dde71 TRANSMISSION_HOME is currently set to: /config/transmission-home [...] Updating Transmission settings.json with values from env variables Attempting to use existing settings.json for Transmission Successfully used existing settings.json /config/transmission-home/settings.json [...] Overriding download-dir because TRANSMISSION_DOWNLOAD_DIR is set to /data/completed Overriding incomplete-dir because TRANSMISSION_INCOMPLETE_DIR is set to /data/incomplete Overriding rpc-password because TRANSMISSION_RPC_PASSWORD is set to [REDACTED] Overriding rpc-port because TRANSMISSION_RPC_PORT is set to 9091 Overriding rpc-username because TRANSMISSION_RPC_USERNAME is set to Overriding watch-dir because TRANSMISSION_WATCH_DIR is set to /data/watch [...] Transmission startup script complete.

HW/SW Environment

- OS: DSM7
- Docker: 20.10.23

Anything else?

The Transmission configuration options docs section says In previous versions of this container the settings were not persistent but were generated from environment variables on container startup. [...] Settings are now persisted in the /config/transmission-home folder in the container and as long as you mount /config you should be able to configure Transmission using the UI as you normally would.

The docs imply that settings ARE persistent, configuring transmission via ENV vars is not necessary, and that I could do everything in settings.json as long as I use the UI or make edits while the container is stopped. So when basic options like download-dir and incomplete-dir were getting overwritten, I was very confused.

The logs did explain they were being overridden by ENV, but my compose file didn't have any TRANSMISSION_* vars set! I was pulling my hair out until I realized the container came with ENV vars baked in that always take precedence unless I shadow them with my own values.

skywayskase commented 4 months ago

Yep, I'm experiencing this same issue as well. Once I restart the container, any settings I set in the previous instance, or manually set in the config file while the container was off, are overwritten unless I specify the settings as env variables when I start the container. I opened a discussion about this a little over a year ago but the issue was never resolved (I think I closed the discussion and just set my env variables and let it be).

pkishino commented 3 months ago

So, just to clarify here, you are seeing that manually setting certain options (only those that we have as necessary defaults in our setup or also others?) get overwritten in settings.json by these defaults if you don’t manually set them as env variables? I think we need to do a deep dive on this and first get a complete list of which variables. I know that we set a number of env variables when the container starts but none of these are UI settings afaik. these settings are there to ensure a usable container is started first time on

skywayskase commented 3 months ago

So, just to clarify here, you are seeing that manually setting certain options (only those that we have as necessary defaults in our setup or also others?) get overwritten in settings.json by these defaults if you don’t manually set them as env variables? I think we need to do a deep dive on this and first get a complete list of which variables. I know that we set a number of env variables when the container starts but none of these are UI settings afaik. these settings are there to ensure a usable container is started first time on

Because several ENV variables are set in the dockerfile, anytime the container is (re)created, those values will overwrite their corresponding settings in the settings.json file, unless different ENV variables are specified at runtime. I rectreate my container all the time, particularly when updating the image.

This was especially an issue because my Transmission RPC credentials kept getting reset to the default (blank) values until I manually specified them as ENV variables in docker run. And because it was behind a proxy and open to the world, that meant anyone who knew the URL had full access and could download whatever they wanted to my server until I figured out the issue and "fixed" it.

phipolis commented 3 months ago

To illustrate @skywayskase's excellent summary:

The Dockerfile sets these ENV TRANSMISSION_* with default values:

https://github.com/haugene/docker-transmission-openvpn/blob/deebad02f47c144ec6551ad5c9fedb3359fb35b1/Dockerfile#L99-L104

updateSettings.py will always overwrite settings.json with values found in ENV vars:

https://github.com/haugene/docker-transmission-openvpn/blob/deebad02f47c144ec6551ad5c9fedb3359fb35b1/transmission/updateSettings.py#L60-L96

So the only way to change those settings is to override the environment variables when starting the container.

lodart commented 1 month ago

Same issue here. I've been banging my head against the wall for hours, now I understand 😌

haugene / docker-transmission-openvpn