search

Breakthrough / PySceneDetect

:movie_camera: Python and OpenCV-based scene cut/transition detection program & library.
https://www.scenedetect.com/
BSD 3-Clause "New" or "Revised" License
3.16k stars 387 forks source link

Avisynth input? #351

Closed Boulder08 closed 1 year ago

Boulder08 commented 1 year ago

Is it possible to use an Avisynth script for input? FFmpeg can decode them, but I'm not sure if raw data can be input directly for analysis or if the module always expects an actual video clip. I'd like to use PySceneDetect for replacing scene change detection by ffmpeg, which is quite flaky it seems.

Breakthrough commented 1 year ago

I did a quick test, and you can use the MoviePy backend to do this, as it uses the ffmpeg executable on your system for this purpose. Right now it will print some warnings about being unable to determine aspect ratio, but these can be ignored for most purposes. This requires you to have MoviePy installed on your system as well as the latest version of PySceneDetect.

This option isn't documented right now since it's under development, but you can try it out as follows with the latest version of PySceneDetect:

scenedetect -i script.avs -b moviepy detect-content

Note this isn't included in Windows builds yet, so you'll need to install PySceneDetect from pip and then also install MoviePy to enable it. When installed correctly, you should see it show up when you run scenedetect version. Aside from the warnings, it seems to work well with the save-images and split-video commands. Feel free to try it out and let me know how it works out for you, and if you run into any other issues.

Boulder08 commented 1 year ago

Thanks for the information!

I did a quick test as well, and got it running. I got a couple of other errors but at least it's scanning right now. In fact that first line is the same error I got when trying to use the API with Avisynth script as video source.

[ERROR:0@0.001] global cap_ffmpeg_impl.hpp:1237 open Could not find decoder for codec_id=192
[ERROR:0@0.001] global cap_ffmpeg_impl.hpp:1286 open VIDEOIO/FFMPEG: Failed to initialize VideoCapture
[ERROR:0@2.657] global cap.cpp:166 cv::VideoCapture::open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.8.0) D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): f:\temp\captures\withnail.avs in function 'cv::icvExtractPattern'
Breakthrough commented 1 year ago

Pixel aspect ratio is queried using OpenCV when the video loads since MoviePy doesn't support it yet, so the same messages will show up currently. In the next release I'll make it so this is only done when required (for save-images). I'll also look if suppressing these warnings is feasible so a better error message can be printed.

Thanks for the report!

Boulder08 commented 1 year ago

One more question regarding the detection: does detect-adaptive do a 2-pass process internally or is it supposed to scan the source twice? I tried it shortly and it (visibly) ran one pass and then printed out the scene change list. This test source I'm using has some luma flicker between frames and triggers false detections easily so I thought the adaptive method might be a best general option to use.

Boulder08 commented 1 year ago

I guess that the automatic calculation of kernel-size is also not working, seems to be None in the debug output.

Breakthrough commented 1 year ago

detect-adaptive used to be a two-pass algorithm, but I made it a single pass in v0.6.2. On the command line, the final scene list is always printed at the end once processing is complete. Regarding kernel size, None means kernel size will be calculated automatically. I should probably update the API to allow disabling it, and make the auto part more explicit.

Boulder08 commented 1 year ago

I possibly found a bug while testing my script.

scenedetect -i f:\temp\captures\malone.avs -b moviepy -o c:\temp detect-adaptive -t 2 -m 64 list-scenes -f test.csv creates a 28 frames long scene as the first one.

Timecode List:,00:00:01.168,00:00:09.051,00:00:27.110,00:00:51.635,00:00:58.392,00:01:59.786,00:02:23.268,00:02:28.940,00:02:34.988,00:02:40.577,00:02:47.125,00:04:38.653,00:05:04.095,00:05:08.725,00:05:17.943,00:05:30.121,00:05:37.295,00:05:43.593,00:05:46.346,00:05:49.224,00:06:03.113,00:06:05.865,00:06:09.369,00:06:16.585,00:06:23.800,00:06:33.560,00:06:38.440,00:06:41.776,00:06:45.697,00:06:48.950 Scene Number,Start Frame,Start Timecode,Start Time (seconds),End Frame,End Timecode,End Time (seconds),Length (frames),Length (timecode),Length (seconds) 1,1,00:00:00.000,0.000,28,00:00:01.168,1.168,28,00:00:01.168,1.168 2,29,00:00:01.168,1.168,217,00:00:09.051,9.051,189,00:00:07.883,7.883 3,218,00:00:09.051,9.051,650,00:00:27.110,27.110,433,00:00:18.060,18.060 4,651,00:00:27.110,27.110,1238,00:00:51.635,51.635,588,00:00:24.524,24.524 5,1239,00:00:51.635,51.635,1400,00:00:58.392,58.392,162,00:00:06.757,6.757 6,1401,00:00:58.392,58.392,2872,00:01:59.786,119.786,1472,00:01:01.395,61.395 7,2873,00:01:59.786,119.786,3435,00:02:23.268,143.268,563,00:00:23.482,23.482 8,3436,00:02:23.268,143.268,3571,00:02:28.940,148.940,136,00:00:05.672,5.672 9,3572,00:02:28.940,148.940,3716,00:02:34.988,154.988,145,00:00:06.048,6.048 10,3717,00:02:34.988,154.988,3850,00:02:40.577,160.577,134,00:00:05.589,5.589 11,3851,00:02:40.577,160.577,4007,00:02:47.125,167.125,157,00:00:06.548,6.548 12,4008,00:02:47.125,167.125,6681,00:04:38.653,278.653,2674,00:01:51.528,111.528 13,6682,00:04:38.653,278.653,7291,00:05:04.095,304.095,610,00:00:25.442,25.442 14,7292,00:05:04.095,304.095,7402,00:05:08.725,308.725,111,00:00:04.630,4.630 15,7403,00:05:08.725,308.725,7623,00:05:17.943,317.943,221,00:00:09.218,9.218 16,7624,00:05:17.943,317.943,7915,00:05:30.121,330.121,292,00:00:12.179,12.179 17,7916,00:05:30.121,330.121,8087,00:05:37.295,337.295,172,00:00:07.174,7.174 18,8088,00:05:37.295,337.295,8238,00:05:43.593,343.593,151,00:00:06.298,6.298 19,8239,00:05:43.593,343.593,8304,00:05:46.346,346.346,66,00:00:02.753,2.753 20,8305,00:05:46.346,346.346,8373,00:05:49.224,349.224,69,00:00:02.878,2.878 21,8374,00:05:49.224,349.224,8706,00:06:03.113,363.113,333,00:00:13.889,13.889 22,8707,00:06:03.113,363.113,8772,00:06:05.865,365.865,66,00:00:02.753,2.753 23,8773,00:06:05.865,365.865,8856,00:06:09.369,369.369,84,00:00:03.503,3.503 24,8857,00:06:09.369,369.369,9029,00:06:16.585,376.585,173,00:00:07.216,7.216 25,9030,00:06:16.585,376.585,9202,00:06:23.800,383.800,173,00:00:07.216,7.216 26,9203,00:06:23.800,383.800,9436,00:06:33.560,393.560,234,00:00:09.760,9.760 27,9437,00:06:33.560,393.560,9553,00:06:38.440,398.440,117,00:00:04.880,4.880 28,9554,00:06:38.440,398.440,9633,00:06:41.776,401.776,80,00:00:03.337,3.337 29,9634,00:06:41.776,401.776,9727,00:06:45.697,405.697,94,00:00:03.921,3.921 30,9728,00:06:45.697,405.697,9805,00:06:48.950,408.950,78,00:00:03.253,3.253 31,9806,00:06:48.950,408.950,10000,00:06:57.083,417.083,195,00:00:08.133,8.133

It might also be better to use 0 as the first frame since that is probably the most common way.