Flood container crashes after 5 seconds

Talanor commented 3 years ago

Type: Bug Report

[ ] Try to follow the update procedure described in the README and try again before opening this issue.

Your Environment

Version used: jesec/flood
Environment name and version:
- Docker version 19.03.13-ce, build 4484c46d9d
- docker-compose version 1.27.4, build unknown
- Firefox 84.0.2 (64b)
- Chromium 87.0.4280.141
Operating system and version:
- Archlinux, current
Torrent client and version: jesec/rtorrent

I run flood and rtorrent via this docker-compose file:

version: '3'

services:
  rtorrent-flood:
    image: jesec/flood
    restart: unless-stopped
    depends_on:
      - rtorrent-XXXX
    volumes:
      - "./data/flood:/home/download/.local/share/flood"
      - "./data/rtorrent/XXXX/home/download:/home/download/XXXX:ro"
      - "./data/rtorrent/XXXX/home/rtorrent.sock:/tmp/socket-XXXX.sock"
    networks:
      - proxy-tier
    environment:
      VIRTUAL_HOST: "YYYYYYYYYY"
      VIRTUAL_PORT: "3000"
      LETSENCRYPT_HOST: "YYYYYYYYYY"

  rtorrent-XXXX:
    image: jesec/rtorrent
    restart: unless-stopped
    tty: true
    volumes:
      - "./data/rtorrent/XXXX/home:/home/download/.local/share/rtorrent:rw"
      - "./data/rtorrent/XXXX/rtorrent.rc:/etc/rtorrent/rtorrent.rc:ro"

networks:
  proxy-tier:
    external:
      name: proxy_proxy-tier

which runs behind a proxy (jwilder/nginx-proxy:alpine and jrcs/letsencrypt-nginx-proxy-companion)

Slightly modified rtorrent.rc https://pastebin.com/Z2TQCPqp

Summary

When I load the page on firefox, I get this in the console about 5 seconds in:

firefox-flood-error

On chromium I get:

chromium-console-flood-error

My proxy then notifies me that the flood container dies, which it does, while logging this absolute unit of a string:

https://pastebin.com/2AaLZejC

Expected Behavior

Flood container not dying

Current Behavior

Flood container dies

Possible Solution

I have little clue as to what to do next since I do not see an appropriate error message. I would guess that flood crashes when the URL https://YYYY/api/activity-stream?historySnapshot=FIVE_MINUTE is accessed ?

Steps to Reproduce

docker-compose up -d
Log in
Wait 5 seconds
Profit

Context

jesec commented 3 years ago

Potentially related: #128

First, try to upgrade to master channel (rolling build). There might be some unreleased changes that will solve the issue.

Second, there are some additional filesystem context requirements if rTorrent is the torrent client. rTorrent should have read access to Flood’s temporary directory. See https://github.com/jesec/flood#troubleshooting. The easiest way is to use a shared HOME directory.

Talanor commented 3 years ago

I will look into it & get back to you on that, but sharing a home between rtorrent and flood is a no-go for me as I am doing a multi user install with one rtorrent instance per-user and sharing the whole folder would be a privacy breach of sorts.

EDIT:

Potentially related: #128

I have checked it and it seems the issue is that the socket is not created due to bad rights given to the folder holding it. Which is not my case, there is no EACCES that I can see and the socket is created properly, the torrents can be created and rtorrent seems to be working fine

First, try to upgrade to master channel (rolling build). There might be some unreleased changes that will solve the issue.

I will try that

Second, there are some additional filesystem context requirements if rTorrent is the torrent client. rTorrent should have read access to Flood’s temporary directory. See https://github.com/jesec/flood#troubleshooting. The easiest way is to use a shared HOME directory.

I fail to see why rtorrent should have write access on /home/download/.local/share/flood/temp since I do not refer to this path anywhere in rTorrent configuration ? I can mount it if you want me to but that looks like weird design if this is the issue.

jesec commented 3 years ago

Troubleshooting section has a detailed explanation on that. Flood downloads/saves torrents to its temporary directory and then adds them to rTorrent by local path.

Talanor commented 3 years ago

Hey again, I tried with master, the crash does not occur at launch but somewhat randomly way farther down the line, I will investigate further and report here. As to

Troubleshooting section has a detailed explanation on that. Flood downloads/saves torrents to its temporary directory and then adds them to rTorrent by local path.

Ok very well, I haven't had the issue yet as I only work with magnet links, I'll give that a go as well.

jesec commented 3 years ago

Try the latest master-dbg image. It is a drop-in replacement for master image.

master-dbg image contains necessary debugging symbols and should have more helpful logs.

Talanor commented 3 years ago

Ok, I have found a reliable way to trigger a crash: Right click on a torrent > Download contents (and I cancel the download after ~ 1s) I see my RAM filling up and eventually flood crashes. Fyi, the content I try to download there is ~60 GB

rtorrent-flood_1    | Error: spawn ENOMEM
rtorrent-flood_1    |     at ChildProcess.spawn (node:internal/child_process:411:11)
rtorrent-flood_1    |     at spawn (node:child_process:617:9)
rtorrent-flood_1    |     at execFile (node:child_process:262:17)
rtorrent-flood_1    |     at node:child_process:194:29
rtorrent-flood_1    |     at Object.linux (/usr/src/app/server/util/diskUsageUtil.ts:32:5)
rtorrent-flood_1    |     at Timeout.DiskUsage.updateDisks [as _onTimeout] (/usr/src/app/server/models/DiskUsage.ts:61:60)
rtorrent-flood_1    |     at listOnTimeout (node:internal/timers:556:17)
rtorrent-flood_1    |     at processTimers (node:internal/timers:499:7)

jesec commented 3 years ago

This is probably related to pipe pressure. Flood archives multiple files into a tar. Obviously your server do that faster than your download speed. Thus, data can linger in memory before they are consumed.

Talanor commented 3 years ago

This is probably related to pipe pressure. Flood archives multiple files into a tar. Obviously your server do that faster than your download speed. Thus, data can linger in memory before they are consumed.

Nice, is there any way to circumvent this ? Related: When I do this manual download, I indeed have a DL speed at about 100 KB/s while if I scp from the same server, I get 10 MB/s (which is my normal download speed). Is there any way this slowing down comes from flood ?

jesec commented 3 years ago

This is probably related to pipe pressure. Flood archives multiple files into a tar. Obviously your server do that faster than your download speed. Thus, data can linger in memory before they are consumed.

Nice, is there any way to circumvent this ? Related: When I do this manual download, I indeed have a DL speed at about 100 KB/s while if I scp from the same server, I get 10 MB/s (which is my normal download speed). Is there any way this slowing down comes from flood ?

Unlikely. I just tested on a crappy machine and I easily get 1000Mbps (100M/s), which is the maximum link speed.

jesec commented 3 years ago

Try the latest build.

Talanor commented 3 years ago

You are way too efficient, damn. It looks like it's working. I'll close this issue and will repost if an error occurs again. Thanks for the work and dedication.

jesec / flood