stashapp / stash

An organizer for your porn, written in Go. Documentation: https://docs.stashapp.cc
https://stashapp.cc/
GNU Affero General Public License v3.0
9.41k stars 804 forks source link

[Feature] Move damaged/untreated video files to an "error" folder #4431

Open gohe007 opened 11 months ago

gohe007 commented 11 months ago

Describe the solution you'd like If there is unexpectedly a damaged or incorrect file in the collection, an attempt is made to index it every time it is scanned/generated. Now it is quite a job to extract those files (via log etc). Is it possible to build in an option where stash moves the files it cannot process into a configurable folder?

Or can this also be solved with a plugin?

UltraHKR commented 10 months ago

Or at least tag the scene and alert the user?

LeGrosFromage commented 10 months ago

Yep, this would be useful. For example, I have a 8K, 2-hr long VR video - so the file size is pretty big - and although it plays back absolutely fine, there's something about it Stash doesn't like and whenever I run a scan or a generate, it spends a lot of time (re)processing that file. If it's the last file in the queue, then fine, I can end the job, but often times it's not so I have to let it do it's thing before it gets to actual new files. The ability to tag it somehow so it's skipped or otherwise recognized as "I processed this once, something ain't right but I'll skip it anyway" would definitely be a nice-to-have.

stg-annon commented 10 months ago

one of the core tenants of stash is that it does not change or modify your files moving the files would go against this, ideally we would want to flag the file/scene either in a dedicated log or DB table, or even via a tag

I think I might work on a plugin for this should be easy enough to parse the existing log or just run ffprobe across the stash to check and flag scenes

LeGrosFromage commented 10 months ago

I think adding a tag, like "fubar" or something, which the scan/generation tasks would recognize as a cue to bypass that scene, would be a very workable solution.

stg-annon commented 10 months ago

I think we should actually lean on the tag system for some of the things in core stash, instead of needing to add specific flags for an object we could add "Service Tags" which are normally hidden from view but are applied by plugins or by stash systems, many plugins already do this but it can look unsightly after enough things are added

stg-annon commented 10 months ago

Here is a simple plugin for this findFileErrors you cant tag most of these files because they tend to not even make it into stash as scenes so there is nothing to tag

I don't really have an example of a damaged file that will play in stash and still leave ffprobe artifacts if I find one I'll update to tag the scene in stash

LeGrosFromage commented 10 months ago

Nice one, I'll take a look.

I have one scene that produces this in the log on a Generate task:

24-01-18 11:58:07
Error   
error generating sprite: ffmpeg command produced no output: <-v error -y -ss 7530.940495592592 -i /StashMedia/Funscripts/VR/Cherry Kiss/[WankzVR] Little Cummer Toy (2019-12-23).mp4 -frames:v 1 -vf scale=160:-2 -c:v bmp -f rawvideo ->

It plays just fine in Stash, but absolutely will not generate the thumbnails/sprites/vtt thingies. If you want I can upload the video to Mega for you to grab, it's 26Gb though.

v0.24.3 Build hash: aeb68a58

stg-annon commented 10 months ago

is that the full error from the logfile? I could use that and add a task for generation errors should be easy enough to tag the scenes

LeGrosFromage commented 10 months ago

Yep that's all I get.

Pompey69 commented 10 months ago

I have a ton of errors thrown up when generating, ffmpeg will not generate the thumbnails/sprites.

error generating sprite: ffmpeg command produced no output: <-v error -y -ss 252.34259259259255 -i /data/Natalia Nemchinova/Natalia A (Danica, Natalia Andreeva) [rus face]/Erotic/HD.mkv -frames:v 1 -vf scale=160:-2 -c:v bmp -f rawvideo ->

I tried to load the findFileErrors plugin (v0.24.3) but I get the error-"no task with name file_check in plugin Find File Errors"

stg-annon commented 10 months ago

I tried to load the findFileErrors plugin (v0.24.3) but I get the error-"no task with name file_check in plugin Find File Errors"

fixed that issue and added in a task for generate errors

kudddyyy commented 9 months ago

Those are the errors I get, would be amazing to catch them too;

2024-02-13 16:52:29
Error   
error processing "H:\\__downloads__\\3DSVR-0621-B.mp4": running ffprobe on "H:\\__downloads__\\3DSVR-0621-B.mp4": FFProbe encountered an error with <H:\__downloads__\3DSVR-0621-B.mp4>.
Error JSON:
{
    "error": {
        "code": -541478725,
        "string": "End of file"
    }
}

Error: exit status 1
2024-02-13 16:52:27
Info    
Calculating fingerprints for H:\__downloads__\3DSVR-0621-B.mp4 ...
2024-02-13 16:52:27
Error   
error processing "H:\\__downloads__\\3DSVR-0621-A.mp4": calculating fingerprint for file "H:\\__downloads__\\3DSVR-0621-A.mp4": calculating oshash: EOF
2024-02-13 16:52:16
Info    
Calculating fingerprints for H:\__downloads__\3DSVR-0621-A.mp4 ...
throwaway2257 commented 8 months ago

If you are generating perceptual hashes, I find that damaged files will generally yield a null phash (assuming you generate a phash for every scene). You can then filter scenes by "PHash is null," verify as necessary, and then move/delete scenes.