debridmediamanager / zurg-testing

A self-hosted Real-Debrid webdav server you can use with Infuse. Together with rclone it can mount your Real-Debrid torrent library into your filesystem and load it to Plex or Jellyfin.
489 stars 38 forks source link

Zurg running OOM during active SCAN and addition of new data #9

Closed JurajNyiri closed 11 months ago

JurajNyiri commented 11 months ago

Zurg has been running OOM for me resulting in plex scan marking everything as deleted. I recently changed my docker engine ram limitation from 1gb to 4gb but it is still running OOM. This only happens while Plex scans tv shows library.

Version

ghcr.io/debridmediamanager/zurg-testing:v0.9.1-hotfix.2

Rclone mount

/Users/jurajnyiri/rclone mount -o modules=iconv,from_code=UTF-8 --config=/Users/jurajnyiri/.config/rclone/rclone.conf \
        --allow-other \
        --poll-interval 15s \
        --log-level=INFO \
        --dir-cache-time 10s \
        --cache-dir="${media}/tmp/rclone_cache_Zurg" \
        --vfs-cache-mode full \
        --vfs-cache-max-size 50G \
        --vfs-cache-max-age 24h \
        --log-file="$media"rclone-mount.log \
        zurg: "${mntpoint}/Zurg" &

Config.yml

# Zurg configuration version
zurg: v1

token: REDACTED # https://real-debrid.com/apitoken

host: "[::]" # do not change this if you are running it inside a docker container
port: 9999 # do not change this if you are running it inside a docker container
concurrent_workers: 200
check_for_changes_every_secs: 15
info_cache_time_hours: 12
serve_from_rclone: true

enable_repair: true # BEWARE! THERE CAN ONLY BE 1 INSTANCE OF ZURG THAT SHOULD REPAIR YOUR TORRENTS
retain_folder_name_extension: false # if true, zurg won't modify the filenames from real-debrid
on_library_update: |
  token="REDACTED"
  basepath="REDACTED"
  for arg in "$@"
  do
    echo "detected update on: $arg"
    if [[ $arg == movies/* ]]; then
      section=14
    elif [[ $arg == shows/* ]]; then
      section=15
    else
      exit 0
    fi
    curl --verbose --get --data-urlencode "path=$basepath/$arg" --data-urlencode "X-Plex-Token=$token" "http://192.168.100.19:32400/library/sections/$section/refresh" 
  done

network_buffer_size: 1048576 # 1 MiB

# curl https://raw.githubusercontent.com/debridmediamanager/real-debrid-network-test/main/real-debrid-network-test.sh | bash

# List of directory definitions and their filtering rules
directories:
  shows:
    group: media
    group_order: 20
    filters:
      - has_episodes: true  # intelligent detection of episode files inside a torrent

  movies:
    group: media
    group_order: 30 
    filters:
      - regex: /.*/ # you cannot leave a directory without filters because it will not have any torrents in it

Logs

zurg.log zurg_memory.log

Time in zurg.log is one our behind the memory log.

JurajNyiri commented 11 months ago

I have tried changing serve_from_rclone: true to false with no effect on OOM.

JurajNyiri commented 11 months ago

I have tried removing on_library_update: | with no effect.

JurajNyiri commented 11 months ago

I stopped plex debrid and the issue is no longer present and scan completed successfully hovering around 300-500mb. It must be an edge case where adding new data during scan causes OOM.

JurajNyiri commented 11 months ago

This happens with no running scan as well with just the on_library_update: . What I did was went to merge the same hash via debrid media manager with the on_library_update being as above.

yowmamasita commented 11 months ago

Please test the latest hotfix image

yowmamasita commented 11 months ago

It should be fixed on 0.9.2 :) 🙏🏼

JurajNyiri commented 11 months ago

Confirming the fix, thank you!!