Scheduled tasks and heavy data usage

[NoSync]

Hi, I've been running pd_zurg for a few months with no issues. I recently moved to a new apartment where I have a monthly data cap, and tonight around 3/4am (which is when runs its maintenance) I recorded a peak in data usage in pd_zurg's container, using over 400 GB in a couple of hours. I double checked on Real Debrid, and sure enough it shows the same amount of data used, so I guess for some reason one of those maintenance tasks triggers the full download of plenty of media from RD. I don't quite understand what causes this though. This is my config file:

version: "3.8"

services:
  pd_zurg:
    container_name: pd_zurg
    image: iampuid0/pd_zurg:latest
    restart: always
    stdin_open: true # docker run -i
    tty: true        # docker run -t    
    volumes:
      ## Location of configuration files. If a Zurg config.yml and/or Zurg app is placed here, it will be used to override the default configuration and/or app used at startup 
      - /home/nosync/media/pd_zurg/config:/config:rw
      ## Location for logs
      - /home/nosync/media/pd_zurg/log:/log:rw
      ## Location for rclone cache if enabled
      - /data/cache/rclone:/cache
      ## Location for Zurg RealDebrid active configuration
      - /home/nosync/media/pd_zurg/RD:/zurg/RD:rw
      ## Location for Zurg AllDebrid active configuration -- when supported by Zurg     
      - /home/nosync/media/pd_zurg/AD:/zurg/AD:rw
      ## Location for rclone mount to host
      - /home/nosync/media/pd_zurg/mnt:/data:rshared  
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Zurich
      ## Zurg Required Settings
      - ZURG_ENABLED=true      
      - RD_API_KEY=XXX
      ## Zurg Optional Settings
      # - ZURG_VERSION=
      - ZURG_UPDATE=false
      - PLEX_REFRESH=true
      - PLEX_MOUNT_DIR=/pd_zurg
      ## Rclone Required Settings
      - RCLONE_MOUNT_NAME=pd_zurg
      ## Rclone Optional Settings - See rclone docs for full list
      - RCLONE_CACHE_DIR=/cache
      - RCLONE_DIR_CACHE_TIME=10s
      - RCLONE_VFS_CACHE_MODE=full
      - RCLONE_VFS_CACHE_MAX_SIZE=32G
     # - RCLONE_VFS_CACHE_MAX_AGE=4h
     # - RCLONE_ATTR_TIMEOUT=8700h
     # - RCLONE_BUFFER_SIZE=32M
     # - RCLONE_VFS_READ_CHUNK_SIZE=32M
     # - RCLONE_VFS_READ_CHUNK_SIZE_LIMIT=1G
     # - RCLONE_TRANSFERS=8
      ## Plex Debrid Required Settings
      - PD_ENABLED=true
      - PLEX_USER=XXX
      - PLEX_TOKEN=XXX
      - PLEX_ADDRESS=http://192.168.50.100:32400
      ## Plex Debrid Optional Settings
      - PD_UPDATE=false
     # - AUTO_UPDATE_INTERVAL=12
     # - SHOW_MENU=false
      ## Special Features
      - DUPLICATE_CLEANUP=false
      - CLEANUP_INTERVAL=24
     # - PDZURG_LOG_LEVEL=DEBUG
     # - PDZURG_LOG_COUNT=2
      - SEERR_API_KEY=XXX
      - SEERR_ADDRESS=http://192.168.50.100:5055
    devices:
      - /dev/fuse:/dev/fuse:rwm
    cap_add:
      - SYS_ADMIN     
    security_opt:
      - apparmor:unconfined    
      - no-new-privileges
    healthcheck:
      start_period: 1m
      interval: 10s
      retries: 3

  plex:
    image: lscr.io/linuxserver/plex:latest
    container_name: plex
    restart: always
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Zurich
      - VERSION=docker
      - PLEX_CLAIM=XXX
#      - DOCKER_MODS=ghcr.io/tailscale-dev/docker-mod:main
#      - TS_NO_LOGS_NO_SUPPORT=true
#      - TAILSCALE_STATE_DIR=/var/lib/tailscale
#      - TAILSCALE_SERVE_MODE=https
#      - TAILSCALE_SERVE_PORT=32400
#      - TAILSCALE_USE_SSH=0
#      - TAILSCALE_HOSTNAME=plex
#      - TAILSCALE_FUNNEL=on
#      - TAILSCALE_AUTHKEY=XXX
    devices:
      - /dev/dri:/dev/dri
      - /dev/fuse:/dev/fuse:rwm
    cap_add:
      - SYS_ADMIN   
    volumes:
      - /home/nosync/media/plex/library:/config
      - /home/nosync/media/plex/subtitles:/subtitles
      - /home/nosync/media/plex/tvseries:/tv
      - /home/nosync/media/plex/movies:/movies
      - /etc/letsencrypt/archive:/ssl
      - /home/nosync/media/plex/tailscale:/var/lib/tailscale
      - /home/nosync/media/pd_zurg/mnt/pd_zurg:/pd_zurg:shared
    ports:
      - 32400:32400
      - 1900:1900/udp
      - 8324:8324
      - 32410:32410/udp
      - 32412:32412/udp
      - 32413:32413/udp
      - 32414:32414/udp
      - 32469:32469
    depends_on:
      pd_zurg:
       condition: service_healthy

Could this be related to RCLONE_VFS_CACHE_MODE or PLEX_REFRESH? Or a specific Plex setting?

For now I moved the stack to its own network bridge and whipped up a script which uses iptables to check the amount of traffic generated by the containers every 15 min and drops down the bridge interface if the threshold is exceeded, but it's obviously less than optimal.

Thanks.

[mattradford]

Hey @NoSync. There is a bandwidth issue described on the plex_debrid repo: https://github.com/itsToggle/plex_debrid/issues/623

Following a comment I looked at Plex settings: https://github.com/itsToggle/plex_debrid/wiki/Setup-Guides#2-setup-plex-2

I can see my bandwidth spiking during the scheduled tasks window:

I have disabled Perform extensive media analysis during maintenance so hopefully that will solve the download issue.

[NoSync]

Hi @mattradford, yep, I also figured out disabling all scheduled tasks was the way to go. It's a bit odd that the peak would occur every time the tasks run and not only the first time (I would imagine the results for the same hash to be cached), but in the meantime this solves it.

Thanks.

[NoSync]

Closing as the issue is on plex_debrid side.