search

StuffAnThings / qbit_manage

This tool will help manage tedious tasks in qBittorrent and automate them. Tag, categorize, remove Orphaned data, remove unregistered torrents and much much more.
MIT License
726 stars 44 forks source link

[Bug]: noHL tag erroneously implied. #375

Closed bakerboy448 closed 9 months ago

bakerboy448 commented 1 year ago

Describe the Bug

hardlinked torrents are tagged as noHL

torrent Miracle.Workers.2019.S04.1080p.AMZN.WEB-DL.DDP5.1.H.264-FLUX

provided 3 log files with trace

relevant log snippets

aker\Downloads\qbit_manage.log (1 hit)
    Line  7123: [2023-08-29 11:39:52,897] [update_share_limits_for_group:190] [TRACE]    | Torrent: Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX [Hash: 5dc677a796212212eb731ec3fc55a9f10a7c805b] |
  C:\Users\baker\Downloads\qbit_manage.log.1.log (1 hit)
    Line 37569: [2023-08-29 11:39:47,568] [assign_torrents_to_group:270] [TRACE]    | Torrent: Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX [Hash: 5dc677a796212212eb731ec3fc55a9f10a7c805b] - Share Limit Group: Default |
  C:\Users\baker\Downloads\qbit_manage.log.2.log (22 hits)
    Line 41215: [2023-08-29 11:39:35,009] [nohardlink:495]            [TRACE]    | Folder: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX  |
    Line 41216: [2023-08-29 11:39:35,009] [nohardlink:496]            [TRACE]    | Files Sorted by size: [PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E09.John.Christ.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E10.The.End.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E06.Olympus.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E05.Jim.Carrey.in.the.Park.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E07.Roland.Proudheart.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E04.The.Grouping.Ceremony.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E01.Welcome.to.Boomtown.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E03.The.MatriXXX.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E08.Children.of.Women.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv'), PosixPath('/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E02.H.O.A.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv')] |
    Line 41217: [2023-08-29 11:39:35,009] [nohardlink:507]            [TRACE]    | Largest file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E09.John.Christ.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41219: [2023-08-29 11:39:35,009] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E09.John.Christ.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41224: [2023-08-29 11:39:35,009] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E10.The.End.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41229: [2023-08-29 11:39:35,010] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E06.Olympus.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41234: [2023-08-29 11:39:35,010] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E05.Jim.Carrey.in.the.Park.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41239: [2023-08-29 11:39:35,010] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E07.Roland.Proudheart.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41244: [2023-08-29 11:39:35,011] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E04.The.Grouping.Ceremony.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41249: [2023-08-29 11:39:35,011] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E01.Welcome.to.Boomtown.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41254: [2023-08-29 11:39:35,011] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E03.The.MatriXXX.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41259: [2023-08-29 11:39:35,012] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E08.Children.of.Women.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |
    Line 41264: [2023-08-29 11:39:35,012] [nohardlink:515]            [TRACE]    | Checking file: /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E02.H.O.A.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv |

file system

bakerboy448@hetz-baker-2004:~$ ls -ilha /data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/
total 8.9G
 14937 drwxrwxr-x   2 bakerboy448 media   12 Aug 29 08:31 .
206808 drwxrwxr-x 532 bakerboy448 media  729 Aug 29 09:55 ..
113340 -rw-rw-r--   3 bakerboy448 media 923M Aug 17 20:41 Miracle.Workers.2019.S04E01.Welcome.to.Boomtown.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
113763 -rw-rw-r--   3 bakerboy448 media 862M Aug 17 20:41 Miracle.Workers.2019.S04E02.H.O.A.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
113341 -rw-rw-r--   3 bakerboy448 media 898M Aug 17 20:41 Miracle.Workers.2019.S04E03.The.MatriXXX.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
113765 -rw-rw-r--   3 bakerboy448 media 924M Aug 17 20:41 Miracle.Workers.2019.S04E04.The.Grouping.Ceremony.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
113856 -rw-rw-r--   3 bakerboy448 media 954M Aug 17 20:42 Miracle.Workers.2019.S04E05.Jim.Carrey.in.the.Park.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
114212 -rw-rw-r--   3 bakerboy448 media 966M Aug 17 20:41 Miracle.Workers.2019.S04E06.Olympus.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
113346 -rw-rw-r--   3 bakerboy448 media 938M Aug 17 20:42 Miracle.Workers.2019.S04E07.Roland.Proudheart.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
113857 -rw-rw-r--   3 bakerboy448 media 878M Aug 17 20:42 Miracle.Workers.2019.S04E08.Children.of.Women.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
  3444 -rw-rw-r--   3 bakerboy448 media 994M Aug 29 07:44 Miracle.Workers.2019.S04E09.John.Christ.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
 15314 -rw-rw-r--   3 bakerboy448 media 984M Aug 28 23:46 Miracle.Workers.2019.S04E10.The.End.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
bakerboy448@hetz-baker-2004:~$ find /data/media -xdev -inum 3444    
/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX/Miracle.Workers.2019.S04E09.John.Christ.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
/data/media/torrents/seeds-tv/Miracle.Workers.2019.S04E09.John.Christ.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX.mkv
/data/media/media/tv/Miracle Workers (2019) {imdb-tt7529770}/Season 04/Miracle Workers (2019) - S04E09 - John Christ [AMZN WEBDL-720p][EAC3 5.1][h264]-FLUX.mkv

Config

#commands:
# The commands defined below will IGNORE any commands used in command line and docker env variables.
#  dry_run: true
#  cross_seed: true
#  recheck: true
#  cat_update: true
#  tag_update: true
#  rem_unregistered: true
#  tag_tracker_error: true
#  rem_orphaned: true
#  tag_nohardlinks: true
#  skip_cleanup: false
#  share_limits: true
#  skip_qb_version_check: false
# qBittorrent parameters
qbt:
  host: 127.0.0.1:10724
  user:
  pass:
settings:
  force_auto_tmm: true
  tracker_error_tag: issue
  nohardlinks_tag: noHL
  share_limits_tag: ~share_limit
  ignoreTags_OnUpdate:
    - noHL
    - issue
    - cross-seed
    - ~share_limit
directory:
  # Do not remove these
  # Cross-seed var: </your/path/here/> # Output directory of cross-seed
  # root_dir var: </your/path/here/> # Root downloads directory used to check for orphaned files, noHL, and RecycleBin.
  # <OPTIONAL> remote_dir var: </your/path/here/> # Path of docker host mapping of root_dir.
  # Must be set if you're running qbit_manage locally and qBittorrent/cross_seed is in a docker
  cross_seed: /data/media/_downloads/torrents.xseed/
  root_dir: /data/media/torrents/
  remote_dir: /data/media/torrents/
  recycle_bin: /data/media/torrents/.RecycleBin
  torrents_dir: /home/bakerboy448/.local/share/qBittorrent/BT_backup
  orphaned_dir: /data/media/torrents/orphaned_data
cat:
  # <Category Name> : <save_path> # Path of your save directory. Can be a keyword or full path
  audiobooks: /data/media/torrents/audiobooks
  books: /data/media/torrents/books
  movies: /data/media/torrents/movies
  movies4k: /data/media/torrents/movies4k
  prowlarr: /data/media/torrents/prowlarr
  tv: /data/media/torrents/tv
  tv4k: /data/media/torrents/tv4k
  seeds-audiobooks: /data/media/torrents/seeds-audiobooks
  seeds-books: /data/media/torrents/seeds-books
  seeds-movies: /data/media/torrents/seeds-movies
  seeds-movies4k: /data/media/torrents/seeds-movies4k
  seeds-tv: /data/media/torrents/seeds-tv
  seeds-tv4k: /data/media/torrents/seeds-tv4k
  seeds-movies.cross-seed: /data/media/torrents/seeds-movies
  seeds-movies4k.cross-seed: /data/media/torrents/seeds-movies4k
  seeds-tv.cross-seed: /data/media/torrents/seeds-tv
  seeds-tv4k.cross-seed: /data/media/torrents/seeds-tv4k
  movies.cross-seed: /data/media/torrents/seeds-movies
  movies4k.cross-seed: /data/media/torrents/seeds-movies4k
  tv.cross-seed: /data/media/torrents/seeds-tv
  seeds-data: /data/media/torrents/seeds-data
  seeds-data.cross-seed: /data/media/torrents/seeds-data
  seeds: /data/media/torrents/seeds/
cat_change:
  # This moves all the torrents from one category to another category. This executes on --cat-update
  # WARNING: if the paths are different and Default Torrent Management Mode is set to automatic the files could be moved !!!
  # <Old Category Name> : <New Category>
  seeds-movies.cross-seed: seeds-movies
  seeds-movies4k.cross-seed: seeds-movies4k
  seeds-tv.cross-seed: seeds-tv
  seeds-tv4k.cross-seed: seeds-tv4k
  tv.cross-seed: seeds-tv
  tv4k.cross-seed: seeds-tv4k
  movies.cross-seed: seeds-movies
  movies4k.cross-seed: seeds-movies4k
  seeds-data.cross-seed: seeds-data

tracker:
  animebytes.tv:
    tag: AnimeBytes
    notifiarr: animebytes
  avistaz:
    tag: Avistaz
    notifiarr: avistaz
  cinemaz:
    tag: Cinemaz
    notifiarr: avistaz
  beyond-hd:
    tag: Beyond-HD
    notifiarr: beyondhd
  blutopia:
    tag: Blutopia
    notifiarr: blutopia
  cartoonchaos:
    tag: CartoonChaos
  digitalcore:
    tag: DigitalCore
    notifiarr: digitalcore
  gazellegames:
    tag: GGn
  hdts:
    tag: HDTorrents
    notifiarr: hdtorrents
  landof.tv:
    tag: BroadcasTheNet
    notifiarr: broadcasthenet
  myanonamouse:
    tag: MaM
  passthepopcorn:
    tag: PassThePopcorn
    notifiarr: passthepopcorn
  privatehd:
    tag: PrivateHD
    notifiarr: privatehd
  tleechreload:
    tag: TorrentLeech
    notifiarr: torrentleech
  torrentleech:
    tag: TorrentLeech
    notifiarr: torrentleech
  tv-vault:
    tag: TV-Vault
  other:
    tag: unknown
share_limits:
  noHL-movies:
    priority: 1
    include_all_tags:
      - noHL
    categories:
      - movies
      - movies4k
      - prowlarr
      - seeds-movies
      - seeds-movies4k
      - seeds-data
    max_ratio: 3
    max_seeding_time: 43200 # 30 Days
    cleanup: true
    resume_torrent_after_untagging_noHL: true
    min_num_seeds: 10
  noHL-tv:
    priority: 2
    include_all_tags:
      - noHL
    categories:
      - tv
      - seeds-tv
      - tv-4k
      - seeds-tv4k
    max_ratio: 3
    max_seeding_time: 14400 # 10 Days
    cleanup: true
    resume_torrent_after_untagging_noHL: true
    min_num_seeds: 5
  MaM-Seeds:
    priority: 3
    categories:
      - seeds-books
    max_ratio: 100
    max_seeding_time: -1
    cleanup: true
    resume_torrent_after_untagging_noHL: true
    min_num_seeds: 10
  Default:
    priority: 999
    max_ratio: -1
    max_seeding_time: -1
    cleanup: false
    resume_torrent_after_untagging_noHL: true
    add_group_to_tag: false
nohardlinks:
  - movies
  - movies4k
  - prowlarr
  - seeds-movies
  - seeds-movies4k
  - tv
  - seeds-tv
  - tv-4k
  - seeds-tv4k
  - seeds-data
recyclebin:
  enabled: true
  empty_after_x_days: 1
  save_torrents: true
  split_by_category: false
orphaned:
  # File patterns that will not be considered orphaned files. Handy for generated files that aren't part of the torrent but belong with the torrent's files
  exclude_patterns:
    - "**/.DS_Store"
    - "**/Thumbs.db"
    - "**/@eaDir"
    - "**/temp/**"
    - "**/*.!qB"
    - "**/_unpackerred"
    - /data/media/torrents/seeds-data/**

  # Apprise integration with webhooks
  # Leave Empty/Blank to disable
  empty_after_x_days: 1
apprise:
notifiarr:
  apikey: (removed)
  instance: MyFlix-Qbit
webhooks:
  error: notifiarr
  run_start: notifiarr
  run_end: notifiarr
  function:
    cross_seed: notifiarr
    recheck: notifiarr
    cat_update: notifiarr
    tag_update: notifiarr
    rem_unregistered: notifiarr
    rem_orphaned: notifiarr
    tag_nohardlinks: notifiarr
    empty_recyclebin: notifiarr
    tag_tracker_error: notifiarr
    share_limits: notifiarr
bhd:
  apikey: (removed)

Logs

https://gist.github.com/bakerboy448/b85c8ebc316ab7bd4250a337b8e0288e | https://gist.github.com/bakerboy448/1874d751e4c09454e8f85556f108281b | https://gist.github.com/bakerboy448/c2421b26aef8f2f0ff6dafa0f643c30b

Screenshots

image

Installation

Nix

Version Number

4.0.3

What branch are you on?

develop

bobokun commented 11 months ago

hmm :( doesn't seem like any of those 3 log files contain the relevant torrent information. I tried looking up the hash 5dc677a796212212eb731ec3fc55a9f10a7c805b but couldn't find it. Miracle.Workers.2019.S04.720p.AMZN.WEB-DL.DDP5.1.H.264-FLUX doesn't have any matches either

Edit: Did some more debugging but I can't seem to reproduce this error. https://discord.com/channels/764440599066574859/921548253948964904/1155221703752962211

fabricionaweb commented 11 months ago

Hi, I was wondering if this was intended to be like that because of cross-seed, if I remember it correctly.

We are just not considering hl if it doesn't have one inode out of the torrents folder.

I read the refered message in the other issue tagged with notifarr.

Any possible solution to this neeeds to be careful to don't be a regression in this regards. For instance checking the cross-seed tag or category.

spyhuntergenral commented 10 months ago

@bakerboy448 Where are your media files living? are they by chance on a remote smb/cifs share?

bakerboy448 commented 10 months ago

Media Files are a mix on SMB (with norbrl) and local zfs

spyhuntergenral commented 10 months ago

hmm, I'm having the same issue, but i don't think its a QBM issue, i think its an underlying filesystem issue. I'm only seeing this with files on a smb share, during my troubleshooting i came across this: see Inode Numbers Section and also this: last post after fiddling around with server settings and mount options i believe the root cause is a combination of this flag nounix (which seems to get applied regards of what a change) and the metadata cache for the mount point.

So whats happening (at least for me) is the client os (QBM) which is mounting the files from an SMB server is assigning temporary inode numbers to the files and not using the server inode numbers, this leads to my observed behavior of - the initial run of QBM everything gets tagged correctly, but on subsequent runs, certain files for some reason are seen as files and not hardlinks. i haven't been able to identify any kind of pattern to this, other than moon phases. my suspicion is that it's got something to do with other app's accessing the files.

My only idea at this point is to move away from a SMB back end.

bakerboy448 commented 9 months ago

closing; haven't reproduced. blame smb?