aetaric / checkrr

Checkrr Scans your library files for corrupt media and replace the files via sonarr and radarr
MIT License
184 stars 8 forks source link

Adjust file checking logic #45

Open TuRDMaN opened 1 year ago

TuRDMaN commented 1 year ago

Sorry if this is already possible, but I didn't see a way to do this.

Checkrr is trying to reacquire many releases for me because of non-video files that it is marking as bad.

I have tried excluding them by file extension, but some of them don't have extensions.

Is there an option in the config file to only check video files, and ignore everything else? If not, could it be added?

Thanks!

aetaric commented 1 year ago

What you are seeing with "non-video" is a result of checkrr using what's called a "magic number" or "file signature" to determine if it's a video, audio, text, or none of the above. By using this check, checkrr is able to skip doing ffprobe checks on very obviously bad media files resulting in faster runs. Realistically, sonarr, radarr, and lidarr (or equiv) shouldn't move non-media files into place alongside video/audio files. They could do so trivially with a File Signature library and it would solve 99% of people's problems.

As it stands today, modifying checkrr to make filetype optional for these lines and make it a config option is no trivial matter, and would result in reduced stats being gathered with the same result occuring.

aetaric commented 1 year ago

I'd suggest altering checkrr's config and disabling arr services to stop it from trying to replace files. Then, remove any problematic files by hand. The webGUI gives you an easy way to export the list of problematic files as a csv, you can then awk out the file column (or parse it some other way) and you have a nice list of files to remove that can be automated if you'd like. The list there is regenerated every run, so you can clean it out, disable arr services, and let it run again to get a fresh list.

TuRDMaN commented 1 year ago

Okay, thanks!

And as an alternative, would it be possible to have an extension whitelist instead of/along with the extension ignore list? Being able to specify only checking known video file extensions would solve all of my issues.

douginoz commented 1 year ago

Realistically, sonarr, radarr, and lidarr (or equiv) shouldn't move non-media files into place alongside video/audio files.

That seems a bit prescriptive - suggesting that only media files should be in the folders that checkrr checks. People have all sorts of non-media files in media folders - metadata files, zips, text, additional graphics, urls etc. They don't prevent any *arr or media player from operating correctly.

It might not be what everyone does, but it's not invalid!

aetaric commented 1 year ago

For reference @TuRDMaN is this only occurring with radarr?

TuRDMaN commented 1 year ago

is this only occurring with radarr?

It happened in my 2 sonarr instances. It kept trying to "reacquire" when it found non-video files (most of which shouldn't have been there anyway and I deleted them). It wouldn't have bothered me except I kept getting spammed with "reacquire" notifications, so I figure it probably shouldn't be doing that unless it was actually a media file.

aetaric commented 1 year ago

That's... very odd. The sonarr code is doing a literal check to make sure the file path is exactly equal to the translated (if needed) path that checkrr sees. The file that is seen on the sonarr side should be the video file. So I'm not sure how this is picking up bad files. I did notice that radarr doesn't include this check which is why I asked. I'll have to patch that soon.

Shadowex3 commented 4 months ago

I've run into this myself. Synology creates metadata files for its indexing service that checkrr can't handle. It may be better to create a whitelist mode where it will only care about specific filetypes for those of us in this situation.

aetaric commented 4 months ago

I've run into this myself. Synology creates metadata files for its indexing service that checkrr can't handle. It may be better to create a whitelist mode where it will only care about specific filetypes for those of us in this situation.

if you are talking about @eadir you can exclude that with:

checkrr:
  ignorepaths:
    - '@eadir'