coanghel / rclone-docker-automount

Simple Python script in a Docker container to auto-mount rclone remote storage.
GNU General Public License v3.0
12 stars 3 forks source link

mounts.json not found #1

Closed Lebowski89 closed 5 months ago

Lebowski89 commented 5 months ago

Hello,

Thank you for this. Rclone runs fine and so does the dashboard. But the initializer cannot find the mounts.json.

Per log file:

2024-05-23 02:03:22,073 - INFO - Rclone is ready.

2024-05-23 02:03:22,073 - ERROR - mounts.json file not found.

2024-05-23 02:03:22,073 - ERROR - Rclone initialization failed

I have tried it with the mounts.json in the root folder and within the config folder with the same result.

Compose (running in Portainer, hence the stack.env for the env file):

services:
  rclone:
    container_name: rclone
    image: rclone/rclone:latest
    networks:
      proxy:
    ports: 
      - "5572:5572"
    env_file:
      - stack.env
    volumes:
      - /appdata/ubuntu/rclone/config:/config/rclone
      - /appdata/ubuntu/rclone/logs:/logs
      - /appdata/ubuntu/rclone/cache:/root/.cache/rclone # WebUI
      - /:/hostfs:rshared
      - /var/cache/rclone:/vfsCache
      - /etc/passwd:/etc/passwd:ro
      - /etc/group:/etc/group:ro
      - /etc/fuse.conf:/etc/fuse.conf:ro
    devices:
      - /dev/fuse:/dev/fuse:rwm
    cap_add:
      - SYS_ADMIN
    security_opt:
      - apparmor:unconfined
    command:
      - rcd
      - --rc-web-gui
      - --rc-web-gui-no-open-browser
      - --rc-addr=:5572
      - --rc-user=
      - --rc-pass=
      - --log-file=/logs/rclone.log
      - --log-level=NOTICE
      - --cache-dir=/vfsCache
    restart: unless-stopped

  rclone_initializer:
    container_name: rclone_initializer
    image: ghcr.io/coanghel/rclone-docker-automount/rclone-init:latest
    networks:
      proxy:
    env_file:
      - stack.env
    depends_on:
      - rclone
    restart: unless-stopped

networks:
  proxy:
    external: true

I have tried adding a volume or bind mount to initializer but it states that it is not allowed.

Additional property mount is not allowed Additional property volume is not allowed

OS = Ubuntu Server 24.04. I do have Rclone installed on the system via apt as well, but have deleted the remote from it just in case.

Any ideas here? Thanks

coanghel commented 5 months ago

You were close on adding the bind/volume mount, but Docker expects a volumes: block in the YAML, not volume: (even if you're only using a single mount)

You need to provide a bind mount for the folder containing the mounts.json to the /app folder in the initializer container:

  rclone_initializer:
...
    volumes:
      - /docker/rclone/initializer:/app

In this example my mounts.json exists as /docker/rclone/initializer/mounts.json on the host and will be available at /app/mounts.json in the container

I've also updated the main readme to include this in the sample compose: https://github.com/coanghel/rclone-docker-automount/tree/master?tab=readme-ov-file#compose-configuration

Lebowski89 commented 5 months ago

Thanks for the quick reply, coanghel.

Have got the initializer folder inside the rclone folder as such:

cache  config  docker-compose.yml  initializer  logs  stack.env

Path:

    volumes:
      - '/appdata/ubuntu/rclone/initializer:/config'

Same result. mounts.json file not found (also tried with / after initializer)

An ls of the directory:

ls /appdata/ubuntu/rclone/initializer
mounts.json
Lebowski89 commented 5 months ago

Okay, I've just tried this under volumes and compose fired up and it still failed:

      - '~/docker/mounts.json:/mount/mounts.json:rw'

Could it be my mounts.json file?

I created it with touch mounts.json and edited via nano.

Content:

[
  {
    "fs": "saltbox:/mnt/unionfs/downloads/torrents/",
    "mountPoint": "/mnt/sftp/qbittorrent",
    "mountOpt": {
      "AllowOther": true
    },
    "vfsOpt": {
      "CacheMode": "off"
  }
]
coanghel commented 5 months ago

You're currently mounting to the /mount/ folder in the initializer container. Change to:

      - '~/docker/mounts.json:/app/mounts.json:rw'
Lebowski89 commented 5 months ago

Progress, looks like one more little hurdle

2024-05-23 05:04:15,411 - INFO - Waiting for rclone service to be available...

2024-05-23 05:04:15,413 - INFO - Rclone is ready.

Traceback (most recent call last):

  File "/app/./rclone_initializer.py", line 67, in <module>

    if initialize():

  File "/app/./rclone_initializer.py", line 61, in initialize

    mount_payloads_data = read_mount_payloads()

  File "/app/./rclone_initializer.py", line 30, in read_mount_payloads

    with open('mounts.json', 'r') as file:

IsADirectoryError: [Errno 21] Is a directory: 'mounts.json'
coanghel commented 5 months ago

It looks like mounts.json is being interpreted as a directory.

  1. Confirm it's actually a file and actually at that location; i.e. we don't have something like ~/docker/mounts.json/mounts.json or ~/docker/rclone/mounts.json going on
  2. Confirm that ~/docker/mounts.json is actually being found by Portainer; I believe the ~ operator in Portainer doesn't by default map to a user's home directory and instead it uses the root user directory. If a non-existent path is passed to Docker, an empty volume will be created with that name (you could confirm this by checking the mounts in Portainer)
    • Just avoid this entirely and supply the full path to mounts.json instead e.g. /home/lebowski89/docker/mounts.json
  3. Try to mount the parent directory instead of the mounts.json directly e.g.
    volumes:
      - /home/lebowski89/docker/rclone:/app:rw
Lebowski89 commented 5 months ago

Yeah, it's just not liking me for some reason.

Compose:

  rclone_initializer:
    container_name: rclone_initializer
    image: ghcr.io/coanghel/rclone-docker-automount/rclone-init:latest
    networks:
      proxy:
    env_file:
      - stack.env
    volumes:
      - '/home/lebowski89/docker:/app:rw'
    depends_on:
      - rclone
    restart: unless-stopped

Path to file: Screenshot_40

Nano of mounts.json: Screenshot_41

Logfile of the container: Screenshot_42

Another oddity I've encountered is if I get the mounts.json via the wget link it is just a massive (text-wise) html like file: Screenshot_43

So I've been making the json file with touch/nano and inputting the info instead.

To see if it was Ubuntu playing up, I deployed it in a Portainer stack on UnRaid and got the same thing. But when I tried doing this volume (with the UnRaid paths on UnRaid):

    volumes:
      - '/mnt/user/appdata/rclone/initializer/mounts.json:/app/mounts.json:rw'

I get this:

2024-05-23 06:45:10,592 - ERROR - Error decoding JSON from the mounts.json file.

I made sure it was a json file and it has correct file permissions for UnRaid. Screenshot_44

coanghel commented 5 months ago

Ah okay, I gave some bad advice, I forgot the python script was also in the /app directory and by mounting the entire directory we lose it. This would be why you get the error in your third screenshot. The right mount would be for just the mounts.json:

- /home/lebowski89/docker/mounts.json:/app/mounts.json

In your unraid situation, it looks like the script is correctly initializing and it is finding the mounts.json file but the json isn’t parsing.

In your first screenshot your mounts.json is missing a curly brace at the end (there should be two before the final bracket.)

Lebowski89 commented 5 months ago

ding ding ding, we have a winner. Addition of a curly brace got it running and trying to mount my remote on both ubuntu/unraid. Thanks for your time, and thanks for the this tool (in both cases, pointing to the mounts.json was the way to go).