search

crazy-max / docker-rtorrent-rutorrent

rTorrent and ruTorrent Docker image
MIT License
503 stars 112 forks source link

rutorrent UI is not showing up but "loading" #281

Closed Ariloum closed 5 months ago

Ariloum commented 1 year ago

Support guidelines

I've found a bug and checked that ...

Description

After updating the rutorrent-crazymax for unraid docker rutorrent shows no more UI (endless "loading" text), but as I can see by the network traffic it still downloading/uploading some torrents

Expected behaviour

entering http://192.168.8.121:8095/ opens the web-ui

Actual behaviour

I'm getting endless "Loading" message instead of web-ui.

Steps to reproduce

I think I didn't updated the rutorrent-crazymax for long like 6-8 months, after update it today to the latest version (Last Update: Oct 25, 2023) ui stopped working

Docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 4
  Running: 1
  Paused: 0
  Stopped: 3
 Images: 4
 Server Version: 20.10.21
 Storage Driver: btrfs
  Build Version: Btrfs v5.10.1 
  Library Version: 102
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1c90a442489720eec95342e1789ee8a5e1b9536f
 runc version: v1.1.4-0-g5fd4c4d1
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.19.17-Unraid
 Operating System: Slackware 15.0 x86_64 (post 15.0 -current)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.04GiB
 Name: Tower
 ID: GW6T:RCDH:BJCE:LPCW:ZYVX:YX5A:FAPL:AXUY:OVMH:GIL7:PTPY:PEL4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Docker Compose config

No response

Logs

tail rutorrent.log 
<fault>
<value><struct>
<member><name>faultCode</name>
<value><i4>-506</i4></value></member>
<member><name>faultString</name>
<value><string>Method 'd.multicall' not defined</string></value></member>
</struct></value>
</fault>
</methodResponse>

Additional info

No response

Ariloum commented 1 year ago

looks like it fixed itself after awhile somehow

tofkentof commented 1 year ago

I have a similar issue. Since last update 4 days ago, rutorrent won't load anymore (or at least I believe it is since the last update). It leads to rtorrent become unhealthy and stops seeding. Closing rutorrent helps rtorrent to recover after a couple of minutes.

When rutorrent is not started or closed, rtorrent seeds properly.

I have tried older tags like 4.2.9-0.9.8-0.13.8 and 4.0-0.9.8-0.13.8 but it didn't help.

I am running this on Synology with 32GB of RAM. I tried to stop all other containers, down to 4 as shown in the docker info below, thinking it could have been a resources problem but it didn't help.

docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 38
  Running: 4
  Paused: 0
  Stopped: 34
 Images: 45
 Server Version: 20.10.23
 Storage Driver: btrfs
  Build Version: Btrfs v4.0
  Library Version: 101
 Logging Driver: db
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs db fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b23a389d8c181697302d163356e97dec04eb8d88
 runc version: 5af893d
 init version: ed96d00
 Security Options:
  apparmor
 Kernel Version: 4.4.302+
 Operating System: Synology NAS

 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 31.32GiB
 Name: NAS
 ID: O3K4:UNEN:B7S5:QDHO:BMYV:M6C2:PZKF:QSTK:QU4D:RMMS:NQNK:6Z3L
 Docker Root Dir: /volume1/@docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Compose

---
name: rtorrent-rutorrent

services:
  geoip-updater:
    image: crazymax/geoip-updater:latest
    container_name: geoip-updater
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    networks:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/geoip:/data"
    env_file:
      - "./geoip-updater.env"
    restart: unless-stopped

  rtorrent-rutorrent:
    image: crazymax/rtorrent-rutorrent:latest
    container_name: rtorrent-rutorrent
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    networks:
      - rtorrent-rutorrent
    expose:
      - "6881/udp"
      - "8000"
      - "8080"
      - "9000"
      - "50000"
    ports:
      - target: 6881
        published: 6881
        protocol: udp
      - target: 8000
        published: 8000
        protocol: tcp
      - target: 8080
        published: 8080
        protocol: tcp
      - target: 9000
        published: 9000
        protocol: tcp
      - target: 50000
        published: 50000
        protocol: tcp
    env_file:
      - "rtorrent-rutorrent.env"
      - ".env"
    volumes:
      - "/volume1/docker/rtorrent:/data"
      - "/volume1/docker/download:/downloads"
      - "/volume1/docker/rtorrent/passwd:/passwd"
      #-  "/volume1/docker/rtorrent/rtorrent/watch:/watch"
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
    restart: unless-stopped

  rtorrent-logs:
    image: bash
    container_name: rtorrent-rutorrent-logs
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    command: bash -c 'tail -f /log/*.log'
    network_mode: none
    depends_on:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/rtorrent/log:/log"
    restart: unless-stopped

networks:
  rtorrent-rutorrent:
    name: rtorrent-rutorrent

The latest entry in rutorrent.log is from 2 months ago.

docker logs as soon as rutorrent is kicked in

stderr
22/11/2023
11:58:54
2023/11/22 11:58:54 [info] 560#560: *576 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: REDACTED, server: , request: "GET /php/getplugins.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm/php-fpm.sock:", host: "rutorrent.REDACTED", referrer: "https://REDACTED"

Radarr and Sonarr, both querying port 8000 on a regular basis, report successful connection when rutorrent is not opened. However they time out when rutorrent is started:

stderr
22/11/2023
12:06:50
2023/11/22 12:06:50 [error] 562#562: *630 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "REDACTED:8000"

I am not sure where to go next. Any help would be greatly appreciated!

tofkentof commented 1 year ago

I've added stop_grace_period: 300s to my compose as I am seeding a very large number of torrents and read that it could lead to some issues. Although I doubt it relates to this one.

I have also added a couple of new env variables but it didn't help:

RU_HTTP_TIME_OUT=300 # was 30 by default
RU_RPC_TIME_OUT=300 # was 5 by default
wkpatrick commented 1 year ago

Ive been seeing a similar issue, where rutorrent is unable to connect to rtorrent for quite a while, before it graduates to taking several minutes to load before failing, and eventually ending in a state where rutorrent loads, but all plugins fail to load and almost all torrents have tracker time out errors.

tofkentof commented 11 months ago

I am still facing the issue. I let the container run for a week without interacting with rutorrent at all hoping time would help (as it did for OP) but sadly it did not.

Any pointers would be really appreciated.

stickz commented 11 months ago

@tofkentof @wkpatrick Are you using torrents with UDP trackers by any chance?

tofkentof commented 11 months ago

@stickz in my .rtorrent.rc, trackers.use_udp.set is commented so I guess I only communicate over tcp?

What is your reasoning? I'm curious :)

Jut to confirm that my problem has been happening for over a month. It impacts only rutorrent. rtorrent behind works fine and seeds properly everything.

stickz commented 11 months ago

What is your reasoning? I'm curious :)

rTorrent has known issues with UDP trackers. This docker image does not implement UDNS. There is also anther issue with tracker scraping that could cause this issue to happen.

I'm trying to figure out the best solution for the problem. If your rTorrent software is timing out after 5 minutes and crashing this problem can be easily resolved with a patch to the docker container. How many torrents approximately are you running?

stickz commented 11 months ago

Also please try docker edge. The author just merged my pull request with stability patches which could resolve this problem. https://github.com/crazy-max/docker-rtorrent-rutorrent/pull/286 docker pull crazymax/rtorrent-rutorrent:edge

crazy-max commented 11 months ago

@stickz I planned to cut a release with your patches. I was thinking of this kind of semantic for versioning: 4.2.9-0.9.8+1-0.13.8+1-r0

+<num> marks changes for patches so we can track changes and user can pin to specific tag.

WDYT?

stickz commented 11 months ago

@crazy-max sounds good to me. I will be submitting more patches in the future.

crazy-max commented 11 months ago

https://github.com/crazy-max/docker-rtorrent-rutorrent/releases/tag/4.2.9-0.9.8_1-0.13.8_1-r0

tofkentof commented 11 months ago

I'm trying to figure out the best solution for the problem. If your rTorrent software is timing out after 5 minutes and crashing this problem can be easily resolved with a patch to the docker container. How many torrents approximately are you running?

Something like 15k.

Also please try docker edge

I've pulled the new image from the tag edge but now rtorrent stays in an unhealthy state.

From rtorrent logs, I can see for each torrent:

Upload unchoked slots adjust; currently:0 adjust:0
Upload unchoked slots adjust; currently:0 adjust:0
Download unchoked slots adjust; currently:0 adjust:0
Download unchoked slots adjust; currently:0 adjust:0
7FD8EE0017EC9108422C8A25FFB379E13CAD681D->download_list: Closing download directly.
7FD8EE0017EC9108422C8A25FFB379E13CAD681D->download_list: Inserting download.
7FD8EE0017EC9108422C8A25FFB379E13CAD681D->download_list: Resuming download: flags:0.
84E7711519E8DD17DCC1A05E074A9A4BD545C726->tracker_list: added tracker (group:0 url:https://REDACTED)

It represents over 140k line of logs since rtorrent was restarted few hours ago.

From docker logs, i can see that the radarr and sonarr time out when trying to connect to rtorrent:

2023/12/17 13:43:29 [error] 763#763: *133 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:13:43:29 +0000] "POST /RPC2 HTTP/1.1" 504 160 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 13:44:34 [error] 761#761: *137 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:13:44:34 +0000] "POST /RPC2 HTTP/1.1" 504 160 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 14:44:14 [error] 763#763: *333 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:14:44:14 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 14:44:20 [error] 762#762: *335 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:14:44:20 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 15:37:38 [error] 763#763: *545 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:37:38 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "XML-RPC.NET"
2023/12/17 15:37:39 [error] 763#763: *545 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:37:39 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "XML-RPC.NET"
2023/12/17 15:44:25 [error] 763#763: *574 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:44:25 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"
2023/12/17 15:44:30 [error] 760#760: *578 connect() to unix:/var/run/rtorrent/scgi.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.113:8000"
192.168.192.1 - - [17/Dec/2023:15:44:30 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Radarr/5.1.3.8246 (alpine 3.18.5)"

I did not have this problem when switching back and forth from different tags (4.0, 4.2.9, ...).

I am not sure what rtorrent is doing at the moment. I don't think it's loading the different torrents because i could see some from different trackers in the rtorrent logs but the trackers show that I'm not seeding anything.

I am tempted to leave like that until tomorrow morning and if it doesn't revert to an healthy state. If not I'll roll back to tag latest and see if that helps.

stickz commented 11 months ago

I am tempted to leave like that until tomorrow morning and if it doesn't revert to an healthy state. If not I'll roll back to tag latest and see if that helps.

Yes, please try tag latest. If that doesn't work, I'll submit a pull request for the tracker scrape patch to the docker container.

tofkentof commented 11 months ago

I reverted to latest but it did not help. However symptoms seem to change (my assumption). Now basically rtorrent cannot announce to any tracker:

1702890213 I 738185956C8B1E7FF1F283F31732AAFB14A6E346->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 219B167AA641B0B810902B06068EDA2300ED4650->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 9FD8FCC02C6090A2E429117CCD39994745BE550C->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I CF23CD39F05895A733881D64B0E742152E999075->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I B39E3CFE6E4B76C289E925A28F6DB4E855E3FC38->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 2033869D84463585FADB43E541CBCCE14E368B5E->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 3E9746BAB324E7E602468B8A4BAB60CF2D181A4D->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 360742B99B188AEE55ADA7891897C0410E074901->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 0F51AD5BE0E5320701B89548EF632FC7101AC6FA->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 9E9C92E4CD83EE28C1B647391567B3B4D1065CA9->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I F05C211747605B1AD52FF65F9777A95CFA785B46->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I CCC52A29DA125BCCB8D081A3BB7FE35801AE2AB8->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)
1702890213 I 25185043DC93631CE33F9AADD0F41DC79013BD4B->tracker_list: failed to connect to tracker (url:REDACTED msg:Timed out)

I've got 16379 lines like this, all with a different hash, which seems to be the number of torrents I've got in rutorrent currently.

I suppose all torrents were parsed and tried to be announced to the trackers but timed out. Although I would expect to see those same lines again with the next reannouncement but rtorrent.log has not been populated for the last 90 minutes.

crazy-max commented 11 months ago

I suppose all torrents were parsed and tried to be announced to the trackers but timed out.

Maybe rate-limit?

tofkentof commented 11 months ago

Maybe rate-limit?

Sorry what do you mean by rate limit?

If you mean upload limit, I do not specify any.

Should I post my .rtorrent.rc file here?

stickz commented 11 months ago

@crazy-max @tofkentof Nope, the problem probably related to #288. This should resolve the tracker issues.

stickz commented 11 months ago

@tofkentof Perhaps try edge now? We just pushed #288, which should resolve the tracker problem. docker pull crazymax/rtorrent-rutorrent:edge

tofkentof commented 11 months ago

just to be sure should I use edge or master? #288 has been merged to master already however no new build was released.

stickz commented 11 months ago

just to be sure should I use edge or master? #288 has been merged to master already however no new build was released.

Try edge. It is the latest commit from master. It's not required to tag a release, to use the changes. https://hub.docker.com/r/crazymax/rtorrent-rutorrent/tags

tofkentof commented 11 months ago

sI've been running on edge for pretty much the last 24 hours. I'm a bit worried because I cannot get back to a state where I was before my initial post: rtorrent was running fine and seeding consistently when rutorrent was not running. Since moving to edge, the container goes from healthy to unhealthy every few hours. Torrents go from seeded to unseeded. Moved initially back to latest but same problem is occuring. Now back again on edge and no improvement.

I'm clueless.

I was considering creating a complete new blank container from latest (or edge) using a new volume for /data (for example: "/volume1/docker/rtorrent2:/data") and restoring only .rtorrent.rc. I would also need to create 2 new volumes to point to the previous container for retrieve all my torrents if I am correct:

"/volume1/docker/rtorrent/rtorrent/.session:/data/rtorrent/.session"
"/volume1/docker/rtorrent/rutorrent/share/torrents:/data/rutorrent/share/torrents"

Although I'm not sure if I need to restore other files for this scenario to work.

My files in my rtorrent folder (with some exceptions that are not relevant here I suppose):

xxx@NAS:/volume1/docker/rtorrent$ find . -regex '.*' | grep -vE '\.session|\.csv$|/share/torrents|\.ico$'
.
./rtorrent
./rtorrent/log
./rtorrent/log/rtorrent.log
./rtorrent/log/tracker.log
./rtorrent/log/storage.log
./rtorrent/watch
./rtorrent/rtorrent-cross-seed.sh
./rtorrent/.rtorrent.rc~
./rtorrent/.rtorrent.rc
./rutorrent
./rutorrent/conf
./rutorrent/conf/users
./rutorrent/conf/users/users
./rutorrent/conf/access.ini
./rutorrent/conf/plugins.ini
./rutorrent/plugins
./rutorrent/plugins-conf
./rutorrent/share
./rutorrent/share/users
./rutorrent/share/users/rtorrent
./rutorrent/share/users/rtorrent/settings
./rutorrent/share/users/rtorrent/settings/erasedata
./rutorrent/share/users/rtorrent/settings/rss
./rutorrent/share/users/rtorrent/settings/rss/cache
./rutorrent/share/users/rtorrent/settings/rss/cache/info
./rutorrent/share/users/rtorrent/settings/trafic
./rutorrent/share/users/rtorrent/settings/trafic/trackers
./rutorrent/share/users/rtorrent/settings/trafic/torrents
./rutorrent/share/users/rtorrent/settings/ratio.dat
./rutorrent/share/users/rtorrent/settings/which.dat
./rutorrent/share/users/rtorrent/settings/loginmgr.dat
./rutorrent/share/users/rtorrent/settings/extsearch.dat
./rutorrent/share/users/rtorrent/settings/rtorrent.dat
./rutorrent/share/users/rtorrent/settings/history_data.dat
./rutorrent/share/users/rtorrent/settings/scheduler.dat
./rutorrent/share/users/rtorrent/torrents
./rutorrent/share/users/rtorrent/tmp
./rutorrent/share/settings
./rutorrent/share/settings/erasedata
./rutorrent/share/settings/rss
./rutorrent/share/settings/rss/cache
./rutorrent/share/settings/rss/cache/info
./rutorrent/share/settings/trafic
./rutorrent/share/settings/trafic/trackers
./rutorrent/share/settings/trafic/torrents
./rutorrent/share/settings/ratio.dat
./rutorrent/share/settings/which.dat
./rutorrent/share/settings/labels
./rutorrent/share/settings/cpu.dat
./rutorrent/share/settings/trackers
./rutorrent/share/settings/theme.dat
./rutorrent/share/settings/cookies.dat
./rutorrent/share/settings/history_data.dat
./rutorrent/share/settings/WebUISettings.dat
./rutorrent/share/settings/loginmgr.dat
./rutorrent/share/settings/scheduler.dat
./rutorrent/share/settings/extsearch.dat
./rutorrent/share/settings/rtorrent.dat
./rutorrent/themes
./rutorrent/rutorrent.log
./geoip
./geoip/GeoLite2-City.mmdb
./geoip/GeoLite2-Country.mmdb
./geoip/geoip-updater.env
./passwd
./passwd/rpc.htpasswd
./passwd/rutorrent.htpasswd
./passwd/webdav.htpasswd
./rtorrent-rutorrent.env

What do you think?

stickz commented 11 months ago

What do you think?

Could rtorrent-cross-seed.sh be causing that problem to happen? Also please provide your .rtorrent.rc file. This will verify tracker scrape delay is working properly.

Since moving to edge, the container goes from healthy to unhealthy every few hours.

This is expected for the torrent session saving process.

  1. Does the container recover after 15 minutes?
  2. Do you have adequate disk space available to save your session files?
  3. Do you know if any of the session files are corrupted? This problem was just fixed recently.
tofkentof commented 11 months ago

I have modified slightly my message above since you wrote your messae :)

I always had rtorrent-cross-seed.sh but I have moved it one level above just to confirm it is not the problem. Will keep you posted. I am also commenting out the line in my .rtorrent.rc that references this script.

.rtorrent.rc (as it was before commenting out rtorrent-cross-seed.sh reference)

# Maximum and minimum number of peers to connect to per torrent
throttle.min_peers.normal.set = 1
throttle.max_peers.normal.set = 100

# Same as above but for seeding completed torrents (-1 = same as downloading)
throttle.min_peers.seed.set = 1
throttle.max_peers.seed.set = 50

# Maximum number of simultanious uploads per torrent
throttle.max_uploads.set = 15

# Global upload and download rate in KiB. "0" for unlimited
throttle.global_down.max_rate.set_kb = 0
throttle.global_up.max_rate.set_kb = 0

# Enable DHT support for trackerless torrents or when all trackers are down
# May be set to "disable" (completely disable DHT), "off" (do not start DHT),
# "auto" (start and stop DHT as needed), or "on" (start DHT immediately)
dht.mode.set = auto

# Enable peer exchange (for torrents not marked private)
protocol.pex.set = yes

# Check hash for finished torrents. Might be usefull until the bug is
# fixed that causes lack of diskspace not to be properly reported
pieces.hash.on_completion.set = yes

# Set whether the client should try to connect to UDP trackers
#trackers.use_udp.set = yes

# Set the max amount of memory address space used to mapping file chunks. This refers to memory mapping, not
# physical memory allocation. Default: 1GB (max_memory_usage)
# This may also be set using ulimit -m where 3/4 will be allocated to file chunks
#pieces.memory.max.set = 1GB

# Alternative calls to bind and ip that should handle dynamic ip's
#schedule2 = ip_tick,0,1800,ip=rakshasa
#schedule2 = bind_tick,0,1800,bind=rakshasa

# Encryption options, set to none (default) or any combination of the following:
# allow_incoming, try_outgoing, require, require_RC4, enable_retry, prefer_plaintext
protocol.encryption.set = allow_incoming,try_outgoing,enable_retry

# Set the umask for this process, which is applied to all files created by the program
system.umask.set = 0022

# Add a preferred filename encoding to the list
encoding.add = UTF-8

# Watch a directory for new torrents, and stop those that have been deleted
directory.watch.added = (cat,(cfg.watch)), load.start
#schedule = watch_directory,5,5,"load.normal=/watch/*.torrent,d.delete_tied="
schedule2 = untied_directory, 5, 5, (cat,"stop_untied=",(cfg.watch),"*.torrent")

# Close torrents when diskspace is low
schedule2 = monitor_diskspace, 15, 60, ((close_low_diskspace,1000M))

# Move finished (no need Autotools/Automove plugin on ruTorrent)
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="
method.set_key=event.download.finished,cross_seed,"execute={/data/rtorrent/rtorrent-cross-seed.sh,$d.name=}"

# Logging
log.open_file = "rtorrent", /data/rtorrent/log/rtorrent.log
log.open_file = "tracker", /data/rtorrent/log/tracker.log
log.open_file = "storage", /data/rtorrent/log/storage.log

log.add_output = "info", "rtorrent"
log.add_output = "critical", "rtorrent"
log.add_output = "error", "rtorrent"
log.add_output = "warn", "rtorrent"
log.add_output = "notice", "rtorrent"
log.add_output = "debug", "rtorrent"

log.add_output = "dht_debug", "tracker"
log.add_output = "tracker_debug", "tracker"

log.add_output = "storage_debug", "storage"

This is expected for the torrent session saving process. Does the container recover after 15 minutes? Do you have adequate disk space available to save your session files? Do you know if any of the session files are corrupted? This problem was just fixed recently.

I'll have to check about the 15 minutes mark. I'm not sure but it seems to take longer.

I have 7TB free so it should be fine :) Before the initial problem, I did not have any session files corrupted. Is there a tool I could script to parse all session files and check for consistency?

stickz commented 11 months ago

It looks like you're missing a bunch of values for your .rtorrent.rc file. You can replace @varriable@ with the correct values. https://github.com/crazy-max/docker-rtorrent-rutorrent/blob/master/rootfs/tpls/etc/rtorrent/.rtlocal.rc#L37-L47

trackers.delay_scrape in particular will fail to launch rTorrent, if you don't have the latest patch installed. You're also missing session saving improvements as well. We increased that from 20M to 3H so you can keep seeding.

tofkentof commented 11 months ago

Thanks for that @stickz.

I've altered .rtorrent.rc file that now looks like that:

throttle.min_peers.normal.set = 1
throttle.max_peers.normal.set = 100
throttle.min_peers.seed.set = 1
throttle.max_peers.seed.set = 50
throttle.max_uploads.set = 15
throttle.global_down.max_rate.set_kb = 0
throttle.global_up.max_rate.set_kb = 0
dht.mode.set = auto
protocol.pex.set = yes
pieces.hash.on_completion.set = yes
protocol.encryption.set = allow_incoming,try_outgoing,enable_retry
system.umask.set = 0022
encoding.add = UTF-8
directory.watch.added = (cat,(cfg.watch)), load.start
schedule2 = untied_directory, 5, 5, (cat,"stop_untied=",(cfg.watch),"*.torrent")
schedule2 = monitor_diskspace, 15, 60, ((close_low_diskspace,1000M))
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="
network.xmlrpc.size_limit.set = 1M
schedule2 = session_save, 1200, 3600, ((session.save))
method.set_key = event.download.inserted, 2_save_session, ((d.save_full_session))
trackers.delay_scrape = true
log.open_file = "rtorrent", /data/rtorrent/log/rtorrent.log
log.open_file = "tracker", /data/rtorrent/log/tracker.log
log.open_file = "storage", /data/rtorrent/log/storage.log
log.add_output = "info", "rtorrent"
log.add_output = "critical", "rtorrent"
log.add_output = "error", "rtorrent"
log.add_output = "warn", "rtorrent"
log.add_output = "notice", "rtorrent"
log.add_output = "debug", "rtorrent"
log.add_output = "dht_debug", "tracker"
log.add_output = "tracker_debug", "tracker"
log.add_output = "storage_debug", "storage"

I have also added the new variable RT_TRACKER_DELAY_SCRAPE to my compose.

I'll monitor how it's going and will report.

tofkentof commented 11 months ago

As an update, the situation is worse now :) rtorrent is not seeding anything, I believe consistently.

in rtorrent.log, I can see that it times out on the announcement for each and every tracker (i've got something like 30 trackers that are suppose to announce).

I have been running on Edge for the last 20 hours maybe.

My assumption is that it never got to announce to any tracker for the last 20 hours.

Anything (logs, cfg, ...) you would like me to provide?

stickz commented 11 months ago

Anything (logs, cfg, ...) you would like me to provide?

Yes, I need to know if you're using any UDP trackers. This docker container doesn't support it properly. Also, try ensuring that IPV6 is disabled at the operating system layer. rTorrent does not function properly with IPV6.

Furthermore, consider limiting the amount of trackers that can announce at once.

network.http.max_open.set = 1024
network.max_open_files.set = 2048
network.max_open_sockets.set = 1024

Lastly, this variable needs to be 16M for the amount of torrents you have. We recently added support for 16M. network.xmlrpc.size_limit.set = 16M

tofkentof commented 11 months ago

I've done everything you mentioned except disabling ipv6 on the hosts (synology distro). Instead I have disabled ipv6 in the container itself by adding:

    sysctls:
      net.ipv6.conf.all.disable_ipv6: 1

Should I disable ipv6 at the host level?

The uptime for the container is 3 hours. Nothing is currently seeding unfortunately.

In rtorrent.log, I can see that each torrent is being parsed (successfully?):

1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I A84D132950124EF464997E88A2C072E711DB4AF1->download_list: Closing download directly.
1703164956 I A84D132950124EF464997E88A2C072E711DB4AF1->download_list: Inserting download.
1703164956 I A84D132950124EF464997E88A2C072E711DB4AF1->download_list: Resuming download: flags:0.
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->tracker_list: added tracker (group:0 url:https://REDACTED)
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->download_list: Closing download directly.
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->download_list: Inserting download.
1703164956 I EED619D24F301FCCB143B80457D425AF089492D6->download_list: Resuming download: flags:0.
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->tracker_list: added tracker (group:0 url:https://REDACTED)
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Upload unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I Download unchoked slots adjust; currently:0 adjust:0
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->download_list: Closing download directly.
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->download_list: Inserting download.
1703164956 I D783F17BC759BC52FD793A9A93E790B21D7E49D5->download_list: Resuming download: flags:0.
1703164956 I 8D8443912CB6DA1D507B0A4E3A1D1BD91C03576D->tracker_list: added tracker (group:0 url:http://REDACTED)

My .rtorrent.rc now looks like this:

throttle.min_peers.normal.set = 1
throttle.max_peers.normal.set = 100
throttle.min_peers.seed.set = 1
throttle.max_peers.seed.set = 50
throttle.max_uploads.set = 15
throttle.global_down.max_rate.set_kb = 0
throttle.global_up.max_rate.set_kb = 0
dht.mode.set = auto
protocol.pex.set = yes
pieces.hash.on_completion.set = yes
trackers.use_udp.set = no
protocol.encryption.set = allow_incoming,try_outgoing,enable_retry
system.umask.set = 0022
encoding.add = UTF-8
directory.watch.added = (cat,(cfg.watch)), load.start
schedule2 = untied_directory, 5, 5, (cat,"stop_untied=",(cfg.watch),"*.torrent")
schedule2 = monitor_diskspace, 15, 60, ((close_low_diskspace,1000M))
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="
network.xmlrpc.size_limit.set = 16M
network.http.max_open.set = 1024
network.max_open_files.set = 2048
network.max_open_sockets.set = 1024
schedule2 = session_save, 1200, 3600, ((session.save))
method.set_key = event.download.inserted, 2_save_session, ((d.save_full_session))
trackers.delay_scrape = true
log.open_file = "rtorrent", /data/rtorrent/log/rtorrent.log
log.open_file = "tracker", /data/rtorrent/log/tracker.log
log.open_file = "storage", /data/rtorrent/log/storage.log
log.add_output = "info", "rtorrent"
log.add_output = "critical", "rtorrent"
log.add_output = "error", "rtorrent"
log.add_output = "warn", "rtorrent"
log.add_output = "notice", "rtorrent"
log.add_output = "debug", "rtorrent"
log.add_output = "dht_debug", "tracker"
log.add_output = "tracker_debug", "tracker"
log.add_output = "storage_debug", "storage"

My compose looks like this:

---
name: rtorrent-rutorrent

services:
  geoip-updater:
    image: crazymax/geoip-updater:latest
    container_name: geoip-updater
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    networks:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/geoip:/data"
    env_file:
      - "./geoip-updater.env"
    restart: unless-stopped

  rtorrent-rutorrent:
    image: crazymax/rtorrent-rutorrent:edge
    container_name: rtorrent-rutorrent
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - RU_HTTP_TIME_OUT=300
      - RU_RPC_TIME_OUT=300
      - RT_SESSION_SAVE_SECONDS=10800
      - RT_TRACKER_DELAY_SCRAPE=true
    networks:
      - rtorrent-rutorrent
    expose:
      - "6881/udp"
      - "8000"
      - "8080"
      - "9000"
      - "50000"
    ports:
      - target: 6881
        published: 6881
        protocol: udp
      - target: 8000
        published: 8000
        protocol: tcp
      - target: 8080
        published: 8080
        protocol: tcp
      - target: 9000
        published: 9000
        protocol: tcp
      - target: 50000
        published: 50000
        protocol: tcp
    env_file:
      - "rtorrent-rutorrent.env"
      - ".env"
    volumes:
      - "/volume1/docker/rtorrent:/data"
      - "/volume1/docker/download:/downloads"
      - "/volume1/docker/rtorrent/passwd:/passwd"
      #-  "/volume1/docker/rtorrent/rtorrent/watch:/watch"
    sysctls:
      net.ipv6.conf.all.disable_ipv6: 1
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
    stop_grace_period: 300s
    restart: unless-stopped

  rtorrent-logs:
    image: bash
    container_name: rtorrent-rutorrent-logs
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    command: bash -c 'tail -f /log/*.log'
    network_mode: none
    depends_on:
      - rtorrent-rutorrent
    volumes:
      - "/volume1/docker/rtorrent/rtorrent/log:/log"
    restart: unless-stopped

networks:
  rtorrent-rutorrent:
    name: rtorrent-rutorrent
stickz commented 11 months ago

sit tight @tofkentof. I have #292 coming. I realized the rTorrent software stack was not being built with the intended flags. Also, c-ares + curl is outdated. This will need to be resolved with anther pull request.

This docker container is lacking maintenance. I don't know if bringing it up to date will resolve your 16K torrent problem though.

stickz commented 11 months ago

@tofkentof please provide your ruTorrent config.php file while we wait for the docker container to be transformed. Ensure it has this line: $throttleMaxSpeed = 327625*1024; // DO NOT EDIT THIS LINE!!! DO NOT COMMENT THIS LINE!!!

2 out of 3 PRs are merged. #295 #296 #297. These bleeding edge improvements should be able to handle 50k of TCP trackers.

tofkentof commented 11 months ago

@stickz the config.php is not exposed outside of the container. I guess it's to be expected?

the line $throttleMaxSpeed = 327625*1024; // DO NOT EDIT THIS LINE!!! DO NOT COMMENT THIS LINE!!! is present.

config.php:

<?php

// for snoopy client
$httpUserAgent = 'Mozilla/5.0 (Windows NT 6.0; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0';
$httpTimeOut = 300;
$httpUseGzip = true;

// for xmlrpc actions
$rpcTimeOut = 300;
$rpcLogCalls = false;
$rpcLogFaults = true;

// for php
$phpUseGzip = false;
$phpGzipLevel = 2;

// Rand for schedulers start, +0..X seconds
$schedule_rand = 10;

// Path to log file (comment or leave blank to disable logging)
$log_file = '/data/rutorrent/rutorrent.log';
$do_diagnostic = true;

// Set to true if rTorrent is hosted on the SAME machine as ruTorrent
$localHostedMode = true;

// Set to true to enable rapid cached loading of ruTorrent plugins
// Required to clear web browser cache during version upgrades
$cachedPluginLoading = false;

// Save uploaded torrents to profile/torrents directory or not
$saveUploadedTorrents = true;

// Overwrite existing uploaded torrents in profile/torrents directory or make unique name
$overwriteUploadedTorrents = false;

// Upper available directory. Absolute path with trail slash.
$topDirectory = '/';
$forbidUserSettings = false;

// For web->rtorrent link through unix domain socket
$scgi_port = 0;
$scgi_host = "unix:///var/run/rtorrent/scgi.socket";
$XMLRPCMountPoint = "/RPC2"; // DO NOT DELETE THIS LINE!!! DO NOT COMMENT THIS LINE!!!
$throttleMaxSpeed = 327625*1024; // DO NOT EDIT THIS LINE!!! DO NOT COMMENT THIS LINE!!!

$pathToExternals = array(
    "php"    => '',
    "curl"   => '',
    "gzip"   => '',
    "id"     => '',
    "stat"   => '',
    "python" => '/usr/bin/python3',
);

// List of local interfaces
$localhosts = array(
    "127.0.0.1",
    "localhost",
);

// Path to user profiles
$profilePath = '/data/rutorrent/share';
// Mask for files and directory creation in user profiles.
$profileMask = 0770;

// Temp directory. Absolute path with trail slash. If null, then autodetect will be used.
$tempDirectory = null;

// If true then use X-Sendfile feature if it exist
$canUseXSendFile = false;

$locale = 'UTF8';

$enableCSRFCheck = false; // If true then Origin and Referer will be checked
$enabledOrigins = array(); // List of enabled domains for CSRF check (only hostnames, without protocols, port etc.). If empty, then will retrieve domain from HTTP_HOST / HTTP_X_FORWARDED_HOST
stickz commented 11 months ago

@stickz the config.php is not exposed outside of the container. I guess it's to be expected?

Config looks good. Please try the latest docker release made 40 minutes ago. These changes completely transform the container with GCC 13.2, PHP 8.2, cURL 8.5.0, c-ares 1.24.0 etc. This is an exponential improvement with the build fixes.

If the rTorrent software is still faulty, we can look into further actions. I just need to know if your ruTorrent loads correctly.

tofkentof commented 11 months ago

Thanks for that. I've pulled the new image latest and will report back.

tofkentof commented 11 months ago

Nothing is being seeded unfortunaly and has never been with the new image. I can't share any log right now but will do tomorrow morning (Europe based). What log/cfg are you after?

I have the feeling since I altered the .rtorrent.rc file with items you suggested I can't seed anymore. Is there any item added that could explain what I'm experiencing? I also added RT_TRACKER_DELAY_SCRAPE=true in the compose file.

stickz commented 11 months ago

Nothing is being seeded unfortunaly and has never been with the new image. I can't share any log right now but will do tomorrow morning (Europe based). What log/cfg are you after?

Is there any item added that could explain what I'm experiencing?

Nope, nothing new added could have caused this to happen. If anything, the docker container is more scalable now for tens of thousands of torrents. If you're able to connect to ruTorrent, it's probably some kind of connectivity issue to the tracker.

If all your trackers time out, then no peers will be detected to seed. I would strongly encourage looking into your configuration. I don't believe at this point it's related specially to the rTorrent software. You're not using UDP trackers. This rules out that problem.

wkpatrick commented 11 months ago

I have been also continuing to run into the issue even while on edge. At least for me, I can watch rtorrent use up 48gb of ram and an 8gb (100%) of swap. This is in a 64 gig total system. This ram usage was happening with the initial issue, was not introduced by any of the recent changes.

I have also been able to replicate this on the same system. I comment out the actual rutorrent service in the docker compose and make a new one with the same settings, and a new folder to mount for /data to ensure no torrents are loaded. I am still getting these issues 100% of the time with this.

stickz commented 11 months ago

I have been also continuing to run into the issue even while on edge. At least for me, I can watch rtorrent use up 48gb of ram and an 8gb (100%) of swap. This is in a 64 gig total system. This ram usage was happening with the initial issue, was not introduced by any of the recent changes.

I have also been able to replicate this on the same system. I comment out the actual rutorrent service in the docker compose and make a new one with the same settings, and a new folder to mount for /data to ensure no torrents are loaded. I am still getting these issues 100% of the time with this.

The recommended course of action is to set pieces.memory.max.set = 4096M and disable swap. It's better to allow rTorrent to use the buffers and caches. It should not be physically "allocating" this kind of memory. I have 128GB of RAM. This works great.

tofkentof commented 11 months ago

I was away for a week but back home now.

As mentioned previously, none of edge or latest tags helped. What worried me is that I couldn't get back to the state I was previously: rtorrent seeding and rutorrent failing to connect to rtorrent.

Yesterday evening I switched back to an older tag: 4.2.9-0.9.8-0.13.8. I kept my config as it is (I didn't try to revert it to what it was before when rtorrent was able to seed properly).

Verdict: rtorrent is seeding again. rutorrent is still not able to connect to rtorrent though.

That's a relief on my side that I'm seeding again but at the same time wondering why none of edge or latest helped rtorrent since the last few merges. Question you will ask yourself no doubt ;)

I could try to revert once more to latest to double check that rtorrent is failing to seed if you want me to.

ewenjo commented 11 months ago

I just ran into the same issue after updating from 4.2.6-0.9.8-0.13.8 to latest. Using the edge version didn't solve it either. Reverting back to 4.2.6-0.9.8-0.13.8 worked though.

tofkentof commented 11 months ago

I just ran into the same issue after updating from 4.2.6-0.9.8-0.13.8 to latest.

Was your issue rtorrent not seeding or rutorrent not connecting to rtorrent or both?

stickz commented 11 months ago

Please update the following line in your .rtorrent.rc file to make ruTorrent work with php version 8.2. A hotfix was distributed for this issue already in the next latest version.

# Initialize plugins
execute2 = {sh,-c,/usr/bin/php82 /var/www/rutorrent/php/initplugins.php rtorrent &}

It's highly recommended to be on latest, if you want to move forward with rTorrent crash and performance fixes.

tofkentof commented 11 months ago

Adding this line did not help unfortunately.

I'm still currently on 4.2.9-0.9.8-0.13.8 and cannot upgrade due to seeding issues confirmed by @ewenjo.

docker logs:

stderr 02/01/2024 09:42:43 2024/01/02 09:42:43 [info] 799#799: *27107 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 192.168.192.1, server: , request: "POST /plugins/rpc/rpc.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm/php-fpm.sock:", host: "192.168.1.113:8080", referrer: "http://192.168.1.113:8080/"
stderr 02/01/2024 09:43:19 2024/01/02 09:43:19 [error] 801#801: *27128 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
stderr 02/01/2024 09:44:05 2024/01/02 09:44:05 [error] 801#801: *27134 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
stderr 02/01/2024 09:44:25 2024/01/02 09:44:25 [error] 800#800: *27140 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"

rutorrent.log last entry is from September.

tofkentof commented 11 months ago

Also I wanted to double check that udp is not used (I created another post to not interfere with the current ideas in case the udp thing is a wrong path) I did a network trace on my container only on udp and can see traffic. I assume that in the trace I see peers and announcement traffic?

Trace after replacing any IP I assumed being a peer by PEER and any fqdn that is related to a private tracker by ANNOUNCE_TRACKER:

10:33:07.437253 IP cpef81d0fad0ae3-PEER.36859 > a5eb52044a96.6881: UDP, length 67
10:33:07.438883 IP PEER.45361 > a5eb52044a96.6881: UDP, length 77
10:33:07.444310 IP PEER.19241 > a5eb52044a96.6881: UDP, length 67
10:33:07.593078 IP a5eb52044a96.6881 > PEER.57431: UDP, length 100
10:33:07.601128 IP PEER.14396 > a5eb52044a96.6881: UDP, length 77
10:33:07.614745 IP PEER.21644 > a5eb52044a96.6881: UDP, length 46
10:33:07.630041 IP PEER.41169 > a5eb52044a96.6881: UDP, length 77
10:33:07.650133 IP PEER.14392 > a5eb52044a96.6881: UDP, length 77
10:33:07.665575 IP one.one.one.one.53 > a5eb52044a96.59195: 30850 1/0/0 PTR hosted-by.leaseweb.com. (81)
10:33:07.666222 IP a5eb52044a96.51718 > one.one.one.one.53: 14299+ PTR? PEER.in-addr.arpa. (42)
10:33:07.690463 IP PEER.7768 > a5eb52044a96.6881: UDP, length 77
10:33:07.701001 IP PEER.50321 > a5eb52044a96.6881: UDP, length 296
10:33:07.767819 IP PEER.32710 > a5eb52044a96.6881: UDP, length 77
10:33:07.809632 IP PEER.6881 > a5eb52044a96.6881: UDP, length 77
10:33:07.846229 IP one.one.one.one.53 > a5eb52044a96.51718: 14299 1/0/0 PTR PEER. (85)
10:33:07.847496 IP a5eb52044a96.48412 > one.one.one.one.53: 3001+ PTR? PEER.in-addr.arpa. (45)
10:33:08.155711 IP one.one.one.one.53 > a5eb52044a96.48412: 3001 NXDomain 0/0/0 (45)
10:33:08.155857 IP a5eb52044a96.44679 > one.one.one.one.53: 3001+ PTR? PEER.in-addr.arpa. (45)
10:33:27.776292 IP PEER.19595 > a5eb52044a96.6881: UDP, length 296
10:33:27.776387 IP a5eb52044a96.6881 > PEER.7866: UDP, length 100
10:33:27.875754 IP a5eb52044a96.37967 > one.one.one.one.53: 38342+ PTR? PEER.in-addr.arpa. (46)
10:33:27.896874 IP PEER.7814 > a5eb52044a96.6881: UDP, length 296
10:33:27.896966 IP a5eb52044a96.6881 > PEER.5991: UDP, length 100
10:33:27.974465 IP PEER.5991 > a5eb52044a96.6881: UDP, length 296
10:33:27.974566 IP a5eb52044a96.6881 > PEER: UDP, length 100
10:33:28.062305 IP PEER.51413 > a5eb52044a96.6881: UDP, length 265
10:33:28.062410 IP a5eb52044a96.6881 > PEER.37321: UDP, length 100
10:33:28.326072 IP one.one.one.one.53 > a5eb52044a96.37967: 38342 1/0/0 PTR PEER. (80)
10:33:28.326502 IP a5eb52044a96.49966 > one.one.one.one.53: 21203+ PTR? PEER.in-addr.arpa. (43)
10:33:28.816978 IP one.one.one.one.53 > a5eb52044a96.49966: 21203 1/0/0 PTR PEER. (87)
10:33:28.817434 IP a5eb52044a96.44652 > one.one.one.one.53: 8657+ PTR? PEER.in-addr.arpa. (45)
10:33:30.063319 IP a5eb52044a96.41222 > one.one.one.one.53: 53570+ AAAA? ANNOUNCE_TRACKER. (34)
10:33:30.063319 IP a5eb52044a96.41224 > one.one.one.one.53: 12366+ A? ANNOUNCE_TRACKER. (34)
10:33:30.077200 IP a5eb52044a96.48412 > one.one.one.one.53: 14589+ AAAA? ANNOUNCE_TRACKER. (27)
10:33:30.077202 IP a5eb52044a96.58351 > one.one.one.one.53: 21969+ A? ANNOUNCE_TRACKER. (27)
10:33:30.082136 IP a5eb52044a96.50450 > one.one.one.one.53: 53990+ AAAA? ANNOUNCE_TRACKER. (34)
10:33:30.082161 IP a5eb52044a96.48142 > one.one.one.one.53: 20438+ A? ANNOUNCE_TRACKER. (34)
10:33:30.088214 IP a5eb52044a96.57648 > one.one.one.one.53: 26555+ A? ANNOUNCE_TRACKER. (34)
10:33:30.088235 IP a5eb52044a96.56517 > one.one.one.one.53: 16402+ AAAA? ANNOUNCE_TRACKER. (34)
10:33:30.759498 IP one.one.one.one.53 > a5eb52044a96.52878: 8705 ServFail 0/0/0 (44)
stickz commented 11 months ago

Adding this line did not help unfortunately.

I'm still currently on 4.2.9-0.9.8-0.13.8 and cannot upgrade due to seeding issues confirmed by @ewenjo.

docker logs:

stderr 02/01/2024 09:42:43 2024/01/02 09:42:43 [info] 799#799: *27107 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 192.168.192.1, server: , request: "POST /plugins/rpc/rpc.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm/php-fpm.sock:", host: "192.168.1.113:8080", referrer: "http://192.168.1.113:8080/"
stderr 02/01/2024 09:43:19 2024/01/02 09:43:19 [error] 801#801: *27128 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
stderr 02/01/2024 09:44:05 2024/01/02 09:44:05 [error] 801#801: *27134 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"
stderr 02/01/2024 09:44:25 2024/01/02 09:44:25 [error] 800#800: *27140 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 192.168.192.1, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket", host: "192.168.1.113:8000"

rutorrent.log last entry is from September.

192.168.192.1 looks like it is configured incorrectly. If it is correct, you may want to check your network configuration.

tofkentof commented 11 months ago

If you are talking about the docker network config, it is created by the compose:

$ sudo docker network inspect --verbose rtorrent-rutorrent
[
    {
        "Name": "rtorrent-rutorrent",
        "Id": "e89104cbe2020733bfa94b6f8184c74c5261921c6f040dee4f181dacd1f0be38",
        "Created": "2023-09-21T11:58:40.079280359+01:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "192.168.192.0/20",
                    "Gateway": "192.168.192.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "984f3f31032a19c399e1f68b2a0de940083ff873b50667857703194e67686613": {
                "Name": "geoip-updater",
                "EndpointID": "07d146d44be191464dbc77a7e8c865864f5a31f423da0e29e7fcfe474f70a549",
                "MacAddress": "02:42:c0:a8:c0:02",
                "IPv4Address": "192.168.192.2/20",
                "IPv6Address": ""
            },
            "a5eb52044a964ec8003e5acd36a1af553ad1abd5112b8f1347416195474f777f": {
                "Name": "rtorrent-rutorrent",
                "EndpointID": "98d114b005b3e55f3eed5a2ae543d30e7cddcd98a16dce899c3c6bf284a8d366",
                "MacAddress": "02:42:c0:a8:c0:03",
                "IPv4Address": "192.168.192.3/20",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "rtorrent-rutorrent",
            "com.docker.compose.project": "rutorrent",
            "com.docker.compose.version": "2.20.2"
        }
    }
]
tofkentof commented 10 months ago

@stickz there is no progress on my side sadly. Problem is still occuring. Would you have any lead? Not sure what else to check. If the above network config is not enough, what else would you like me to produce? Thanks!

robzeta commented 9 months ago

Im also having problems, cant get it to work with a newer version than [4.2.9-0.9.8-0.13.8-r0]. Get msg at startup: "No connection to rTorrent. Check if it is really running".

I also tried the latest version as a clean install, works clean but as soon I try to copy the session folder from my old install I get error. Maybe is has something to do with some of the updates in version [4.2.9-0.9.8_1-0.13.8_1-r0]?

(https://github.com/crazy-max/docker-rtorrent-rutorrent/releases/tag/4.2.9-0.9.8_1-0.13.8_1-r0) rTorrent patches Avoid stack overflow for lockfile buffer Increase maximum SCGI request to 16MB Fix saving session files Fix a common rtorrent xml-rpc crash when trying to queue an invalid task Resolve xmlrpc logic crash libtorrent patches Allow 10 gigabit speed throttles

stickz commented 9 months ago

Im also having problems, cant get it to work with a newer version than [4.2.9-0.9.8-0.13.8-r0]. Get msg at startup: "No connection to rTorrent. Check if it is really running".

I also tried the latest version as a clean install, works clean but as soon I try to copy the session folder from my old install I get error. Maybe is has something to do with some of the updates in version [4.2.9-0.9.8_1-0.13.8_1-r0]?

Please try docker edge and report back. There is a critical fix in here for an rTorrent memory alignment crash. This could be happening when hash checking torrents. docker pull crazymax/rtorrent-rutorrent:edge

robzeta commented 9 months ago

Im also having problems, cant get it to work with a newer version than [4.2.9-0.9.8-0.13.8-r0]. Get msg at startup: "No connection to rTorrent. Check if it is really running". I also tried the latest version as a clean install, works clean but as soon I try to copy the session folder from my old install I get error. Maybe is has something to do with some of the updates in version [4.2.9-0.9.8_1-0.13.8_1-r0]?

Please try docker edge and report back. There is a critical fix in here for an rTorrent memory alignment crash. This could be happening when hash checking torrents. docker pull crazymax/rtorrent-rutorrent:edge

I tried :edge now, it dident work at first got the regular error msg (no connection to rTorrent) in webui. But this time I tried to wait a longer then I did earlier. Just when I was to give up after about 15min the webui finally loaded!

So I went ahead and tried the same thing with :latest, same thing there it works but it takes like 15min before the webui is accesseable, that havent been a thing in the earlier releases.

Its also not only the first launch that takes time, when restarting the container I have to wait for like 15min before I can access webui.

Conclusion: It works with :both egde and :latest, but the startup time for webui is like 15min compared to the earlier releases that was accessable almost instatly after reboot