search

RatingPosterDB / rpdb-folders

Monitors Media Folders and Adds Images with Ratings (poster.jpg / background.jpg) from the RPDB API
GNU Lesser General Public License v2.1
23 stars 2 forks source link

[Issue/Bug] Probed media files don't get properly deleted/closed (held by the system) #37

Closed ghost closed 2 years ago

ghost commented 2 years ago

[Issue/Bug] Probed media files don't get properly deleted/closed (held by the system)

Describe the bug This is a weird one (and a long issue report), and I've actually had it for quite a while. This could easily not be the fault of rpdb, but I'll try and explain in detail. To preface the issue, I think I need to describe the way my setup is working. It's basically made from the thoughts of THIS GitHub.

My setup is using mergerfs to combine two folders: 1) /home/appbox/private/local 2) /home/appbox/m/gcd/plex For the sake of easiness, I'll refer to the local folder as /local and the mounted cloud drive as /m/gcd/plex

The first folder is for local storage, and the 2nd is cloud storage (mounted using rclone). I'm using this as the mergerfs fusing: /usr/bin/mergerfs /home/appbox/private/local:/home/appbox/m/gcd/plex /home/appbox/gmedia -o rw,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=auto-full,dropcacheonclose=true

They're combined to a third combined mergerfs folder: /home/appbox/gmedia

Whenever I download anything, I point it towards the /gmedia folder. The way it's set up, it always writes to the /local folder first. Both Radarr, Sonarr, Bazarr and so on is setup to point towards the /gmedia folder. So whenever something gets imported or downloaded, the files will actually get written to the /home/appbox/private/local as per the mergerfs settings.

This is a nice way to set it up, since I also use a daily upload script, that uploads files to the cloud drive. The daily upload script is using the rclone move command, which basically removes the file from the /local folder, whenever the upload of the file is completed.

Which means it goes from /local to /m/gcd/plex - but to the /gmedia folder, it's all the same basically, since the two folders are combined "as one".

RPDB is also configured to use the /gmedia folder, which works awesomely. It reads new files added by Radarr and Sonarr without error and is also able to probe new files (which are technically saved to the /local folder), but it's also able to probe the files already present on the cloud drive (technically /m/gcd/plex) So basically, it works by mergerfs "tricking" the system to believe all files are present at the same time, whether or not they're actually uploaded yet.

RPDB is setup to probe the 4K files and automatically add badges. That works perfectly and without fail. Can't repeat too many times how much I appreciate that feature :smile:

Anyways! The issue is related to said probing of the 4K files. Whenever the beforementioned daily upload script ends, and all the files have been removed by rclone, there's a lot of leftover "ghost files" - sorry for the lack of a better term. The "ghost files" are some form of hardlinks/softlinks (?) that haven't truly been removed from the disk. They still take up the space, until you restart the rpdb-folders app.

If you use the command sudo lsof +L1 (manpage link), that shows open files that have been unlinked, you can see a list of the files probed by the rpdb-folders app. The output looks like this, after a completed upload by the mentioned upload script / rclone move command:

COMMAND       PID     USER   FD   TYPE DEVICE    SIZE/OFF NLINK      NODE NAME
php-fpm7.     820     root    3u   REG    9,2           0     0  52166658 /tmp/.ZendSem.sQkIr4 (deleted)
unattende     907     root    3w   REG    9,2         452     0 210633852 /var/log/unattended-upgrades/unattended-upgrades-shutdown.log.1 (deleted)
mergerfs  3352623     root    6r   REG    9,2 40234101828     0 126222361 /home/appbox/private/local/4KMovies/X-Men - First Class (2011) [tmdb-49538]/X-Men.First.Class.2011.2160p.UHD.BluRay.REMUX.DTS-HD.MA-5.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root   11r   REG    9,2 51855154785     0 125960456 /home/appbox/private/local/4KMovies/Mulan (1998) [tmdb-10674]/Mulan.1998.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root   12r   REG    9,2 54149344186     0 126222490 /home/appbox/private/local/4KMovies/G.I. Joe - The Rise of Cobra (2009) [tmdb-14869]/G.I.Joe.The.Rise.of.Cobra.2009.DV.2160p.UHD.BluRay.REMUX.DTS-HD.MA-5.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root   18r   REG    9,2 42273126640     0 126222439 /home/appbox/private/local/4KMovies/Resident Evil (2002) [tmdb-1576]/Resident.Evil.2002.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root   23r   REG    9,2 49911337924     0 125832098 /home/appbox/private/local/4KMovies/Early Man (2018) [tmdb-387592]/Early.Man.2018.DV.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root   30r   REG    9,2 62268057718     0 125960196 /home/appbox/private/local/4KMovies/Harry Potter and the Half-Blood Prince (2009) [tmdb-767]/Harry.Potter.and.the.Half-Blood.Prince.2009.PROPER.2160p.UHD.BluRay.REMUX.DTS-X-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root   40r   REG    9,2 60005580933     0 133695273 /home/appbox/private/local/4KMovies/Daughters of Darkness (1971) [tmdb-46175]/Daughters.of.Darkness.1971.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root  103r   REG    9,2 82588691949     0 126222609 /home/appbox/private/local/4KMovies/Saving Private Ryan (1998) [tmdb-857]/Saving.Private.Ryan.1998.DV.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.h265-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root  116r   REG    9,2 41285461081     0 126224699 /home/appbox/private/local/4KMovies/Resident Evil - Apocalypse (2004) [tmdb-1577]/Resident.Evil.Apocalypse.2004.EXTENDED.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root  127r   REG    9,2 70846905011     0 126354645 /home/appbox/private/local/4KMovies/V for Vendetta (2006) [tmdb-752]/V.for.Vendetta.2006.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root  172r   REG    9,2 39326778766     0 133694073 /home/appbox/private/local/4KMovies/Resident Evil - Retribution (2012) [tmdb-71679]/Resident.Evil.Retribution.2012.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root  176r   REG    9,2 44119791007     0 126224841 /home/appbox/private/local/4KMovies/Resident Evil - Extinction (2007) [tmdb-7737]/Resident.Evil.Extinction.2007.2160p.UHD.BluRay.REMUX.TrueHD.Atmos-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)
mergerfs  3352623     root  179r   REG    9,2 72007282481     0 126225120 /home/appbox/private/local/4KMovies/The Croods (2013) [tmdb-49519]/The.Croods.2013.2160p.UHD.BluRay.REMUX.DTS-X-7.1.HDR.HEVC-RIPPEDBYMYSELF.mkv (deleted)

The only way remove the open, yet unlinked, files is to restart the rpdb-folders app. I solved this by just restarting the rpdb-folders app a minute or two before the daily upload script starts, which works wonderfully (lol), but to be fair, is a shoddy solution.

To Reproduce

  1. Install rclone
  2. Config a Google Cloud Drive based mount, using rclone config. Let's call it cloudDrive
  3. Setup two folders, using mergerfs - using said settings: /usr/bin/mergerfs /localDriveFolder:/cloudDriveMount /gmedia -o rw,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=auto-full,dropcacheonclose=true
  4. Get some video files, and put them into the /localDriveFolder
  5. Setup rpdb-folders to probe video files from the combined /gmedia folder
  6. Let it probe the files, technically being located in the /localDriveFolder
  7. Use rclone move /localDriveFolder cloudDrive: -vv to move/upload the files from the local folder, to the cloudDrive that we configured in step 2
  8. Wait for the upload to complete
  9. Use sudo lsof +L1 and see that the files are still considered "opened" by the system, yet unlinked.
  10. Be sad and then restart the rpdb-folders app
  11. Be happy that the files are now closed properly, but yet still sad, since it's a shoddy solution

Extra notes I haven't actually tested whether or not this happens with or without using mergerfs. It might also NOT properly close the probed files, when just using a normal FS.. Just haven't tested that. No idea whether or not this issue is related: https://github.com/buzz/mediainfo.js/issues/89

Expected behavior The files getting properly closed after the probing is done.

Operating System Ubuntu 21.10

RPDB Version v0.1.8

jaruba commented 2 years ago

I think you're right, it's very likely that RPDB Folders never closes the files. This should be easily fixed though.

jaruba commented 2 years ago

@VP-EN I wasn't able to reproduce this, but I believe it should close the file handlers now, can you test the new 0.1.9 version?

ghost commented 2 years ago

Not sure if I'll be able to test it today, but I'll test it one of the coming days hopefully :)

jaruba commented 2 years ago

@VP-EN

That's fine, whenever u find the time, tell me if u need me to reset ur API limit.

You might also be interested in some of the other new features that came with this update: sonarr / radarr webhooks and plex connection. The folder name matching was also improved. (although is not used / needed when radarr / sonarr / plex are connected)

ghost commented 2 years ago

@VP-EN

That's fine, whenever u find the time, tell me if u need me to reset ur API limit.

Seems to have worked! I tested two random files, after they downloaded. Just ran a full scan immediatly, "forcing" the probe. No files shows up using sudo lsof +L1. So I'll consider it fixed with the end() line 😄 . Great work!

You might also be interested in some of the other new features that came with this update: sonarr / radarr webhooks and plex connection. The folder name matching was also improved. (although is not used / needed when radarr / sonarr / plex are connected)

Ooh I'll check that out. I would LOVE to use the automatic Plex metadata refresh, but.. I'm not so sure that the Plex Metadata Refresh would be useful to me, since my RPDB is on my download server. I have two servers, one for downloading and one for strictly running Plex off of. The downloading server uploads everything it has downloaded, the following day, which the Plex server then reads off of. This is nice, because then the movies already come uploaded to Plex, with the poster already included. And it's also easier to probe the 4K files, when they're on-disk, instead of probing them when they're on cloud storage. It's possible, albeit very slow and sometimes janky.

So if the plex metadata refreshes the item, as soon as the poster has been downloaded, it won't really do anything, since the Plex server is on another server, where the updated poster.jpg hasn't been "uploaded" to yet... If that makes any sense.

Not sure if there's any solution to that, except like adding an option to make a user-defined / custom "refresh Plex metadata delay" for all the items it has refreshed...

But I'll check it out regardless 😄

Edit Or - maybe I should just move my RPDB to the Plex server, and let the metadata refresh thing work everything out. That could solve the "missing posters, when new movies has been uploaded"-issue. Would the metadata refresh work, if you did a full scan / full overwrite scan? Would it then refresh all the items invidually? That could work for me, I think