PBH-BTN / PeerBanHelper

Automatically block unwanted, leeches and abnormal BT peers with support for customized and cloud rules.| BT 反吸血工具 - 自动封禁不受欢迎、吸血和异常的 BT 客户端,并支持自定义规则。支持 qB/qBEE/Deluge/BiglyBT/Azureus/Vuze/BitComet
GNU General Public License v3.0
2.61k stars 82 forks source link

[BUG] QB Memory Leak问题 #335

Closed CXwudi closed 2 months ago

CXwudi commented 2 months ago

版本号 - Version

v6.0.1

操作系统平台和系统架构 - OS and CPU Arch

Debian 12, X86

部署方式 - Deploy method

官方Docker镜像

关联的下载器类型 - Downloader Type

qBittorrent

问题描述 - Issue Description

有好几次我发现qbittorrent在下载torrent的时候memory usage会不断上升直到整个Debian server都crash了。

后来通过问题https://github.com/qbittorrent/qBittorrent/issues/18830 得到启发,当把PeerBanHelper关掉后,qbittorrent的memory usage才开始趋于稳定。

复现步骤 - Reproduce steps

使用以下docker compose文件启动可复现实例:

services:
  qbittorrent-app:
    image: lscr.io/linuxserver/qbittorrent
    container_name: qbittorrent-app
    restart: unless-stopped
    network_mode: host # to eliminate double-nat and make upnp working
    environment:
      #- PAGID=10000
      - PGID=1000
      - PUID=1000
      - WEBUI_PORT=3902
    volumes:
      - qbittorrent-config:/config
      - ./downloads/:/downloads
    deploy:
      resources:
        limits:
          memory: 0.5g # try to prevent mem leak

  peerbanhelper:
    image: ghostchu/peerbanhelper
    restart: unless-stopped
    container_name: qbittorrent-peerbanhelper
    volumes:
      - peerbanhelper-data:/app/data
    ports:
      - "3903:9898"
    extra_hosts:
      - "host.docker.internal:host-gateway" # allow to access services on host
    environment:
      - PUID=1000
      - PGID=1000

然后通过localhost:3903登录PeerBanHelper,并设置好对qbittorrent的连接,因为qbittorrent有network_mode: host,所以对于PeerBanHelper来说host.docker.internal:3902 是qbittorrent的API网址

其他一切都保持默认设置

之后下载任意种子,观察内存使用。

截图/日志文件 - Screenshot / Logs

开始下载前: image 下载了一集动漫之后Debian的内存占用从4.5G升到4.65G: image

并一直保持在4.65G: image

额外信息 - Addition Information

此问题可能跟qbittorrent的WebUI memory leak有关,详见https://github.com/qbittorrent/qBittorrent/issues/18830 workaround很简单,就是logout再login以释放session

希望PeerBanHelper也可以给个选项或加个功能可以时不时的logout再login以释放session

检查清单 - Check list

CXwudi commented 2 months ago

Translation of this issue for English speakers:


qBittorrent Memory Leak When Using PeerBanHelper

Version

v6.0.1

OS and CPU Arch

Debian 12, X86

Deploy method

Official Docker Image

Downloader Type

qBittorrent

Issue Description

Several times, I've noticed qBittorrent's memory usage continuously increasing while downloading torrents, eventually leading to the entire Debian server crashing.

Inspired by the issue https://github.com/qbittorrent/qBittorrent/issues/18830, I discovered that after disabling PeerBanHelper, qBittorrent's memory usage stabilized.

Reproduce steps

Use the following docker-compose file to launch a reproducible instance:

services:
  qbittorrent-app:
    image: lscr.io/linuxserver/qbittorrent
    container_name: qbittorrent-app
    restart: unless-stopped
    network_mode: host # to eliminate double-nat and make upnp working
    environment:
      #- PAGID=10000
      - PGID=1000
      - PUID=1000
      - WEBUI_PORT=3902
    volumes:
      - qbittorrent-config:/config
      - ./downloads/:/downloads
    deploy:
      resources:
        limits:
          memory: 0.5g # try to prevent mem leak

  peerbanhelper:
    image: ghostchu/peerbanhelper
    restart: unless-stopped
    container_name: qbittorrent-peerbanhelper
    volumes:
      - peerbanhelper-data:/app/data
    ports:
      - "3903:9898"
    extra_hosts:
      - "host.docker.internal:host-gateway" # allow to access services on host
    environment:
      - PUID=1000
      - PGID=1000

Then, log in to PeerBanHelper via localhost:3903 and configure the connection to qBittorrent. Since qBittorrent uses network_mode: host, the API address for PeerBanHelper is host.docker.internal:3902.

Leave all other settings as default.

Afterward, download any torrent and observe the memory usage.

Screenshot / Logs

Before downloading: image

After downloading one anime episode, Debian's memory usage increased from 4.5GB to 4.65GB: image

And it remains at 4.65GB: image

Addition Information

This issue might be related to qBittorrent's WebUI memory leak, as detailed in https://github.com/qbittorrent/qBittorrent/issues/18830. A simple workaround is to log out and then log back in to release the session.

Hopefully, PeerBanHelper can also provide an option or feature to periodically log out and log back in to release the session.

Check list

Ghost-chu commented 2 months ago

Our qBittorrent adapter obviously reuses login sessions. We only use one session unless the session expires. This shouldn't cause a memory leak, since we don't open new sessions indefinitely. If this causes a memory leak, I think qBittorrent is responsible for it.

Frequent login/logout may have a negative impact on the downloader's performance.

Consider that I am also running an instance of qBittorrent (along with PeerBanHelper) for a month. I have not observed any memory leaks, and this Issue does not make a strong case for memory leaks due to API sessions. So I'm leaning more towards other issues.

This Issue seems to me to be just a random Issue from qBittorrent's Issues that I'm looking for and associating with a potentially relevant Issue. This behavior is like trying to find a sword in a boat.

I wouldn't consider investigating it in depth or implementing mitigations until there is further compelling evidence (as it works fine on my device and many other users' devices). A memory leak alone does not prove that this is related to PeerBanHelper.


我们的 qBittorrent 适配器显然会复用登录会话。除非会话过期,否则我们仅使用一个会话。 这不应该造成内存泄露,因为我们并没有无限制的打开新的会话。如果这造成了内存泄漏,我想 qBittorrent 应对此负责。

频繁的登录/登出可能对下载器的性能存在负面影响。

考虑到我也用在运行一个 qBittorrent 实例(与 PeerBanHelper 一同),并且持续运行了一个月。我没有观察到有内存泄露的情况发生,且此 Issue 无法有力的证明是 API 会话导致的内存泄漏。因此我更倾向于是其他问题。

这个 Issue 在我看来只是从 qBittorrent 的 Issues 中随机寻找了一个可能有关的Issue,并与其关联起来。这种行为无异于刻舟求剑。

在有进一步的有力证据之前,我不会考虑深入调查它或者执行缓解措施(因为它在我的设备和许多其它用户的设备上运行良好)。单纯的内存泄露不能证明这就和 PeerBanHelper 有关。

CXwudi commented 2 months ago

否则我们仅使用一个会话。

如果是这样的话那确实没有https://github.com/qbittorrent/qBittorrent/issues/18830 里提到问题。这两天我再多观察观察

这个 Issue 在我看来只是从 qBittorrent 的 Issues 中随机寻找了一个可能有关的Issue,并与其关联起来。这种行为无异于刻舟求剑。

qbittorrent的WebUI memory leak问题不止一个,也包括https://github.com/qbittorrent/qBittorrent/issues/20675 ,这些issue都与interacting with WebUI/API有关系,所以一个巴掌拍不响,当然qbittorrent应该修复这些问题

CXwudi commented 2 months ago

有可能使用v1版的libtorrent qbittorrent,或者libtorrent v2版的qbittorrent进高级设置把Disk IO type设置成POSIX-compliant可行

详见:https://github.com/qbittorrent/qBittorrent/issues/20925 https://github.com/arvidn/libtorrent/issues/6667


Update for English speakers:

You might want to use qbittorrent with v1 version of libtorrent. If not willing to switch, go to your advanced setting in qbittorrent and set Disk IO type to POSIX-compliant

See links of two issues above for more info

CXwudi commented 2 months ago

I am quite confidence that the QB memory leak problem is due to libtorrent v2 MMAP implementation, closing this as unrelated.

Also see https://github.com/arvidn/libtorrent/issues/7551 and https://github.com/qbittorrent/qBittorrent/issues/19988#issuecomment-1845966694

bacon-cheeseburger commented 2 months ago

I am quite confidence that the QB memory leak problem is due to libtorrent v2 MMAP implementation, closing this as unrelated.

Also see arvidn/libtorrent#7551 and qbittorrent/qBittorrent#19988 (comment)

I use libtorrent v1 and has the memory leak problem that results in eventual total system freeze. See https://github.com/qbittorrent/qBittorrent/issues/21068

CXwudi commented 2 months ago

@bacon-cheeseburger I noticed several memory leak issues reported around API and WebUI in qBittorrent. I used to think that I was experiencing the same issue, particularly https://github.com/qbittorrent/qBittorrent/issues/18830. That's why I was reporting the issue to this repo instead of qbittorrent to hope the author can fix any potential session misusage (which in fact doesn't have any).

Then I dive in more and found out the MMAP issues reported from various ppl. I was able to work around it by setting Disk IO type to POSIX-compliant.