wildcards produce different detection than file by file run

[Solarbytes]

Bug/Issue Description: EDITED - FPS was correctly detected by DVR-scan, older clips did not have that discrepancy which led me to think that the frame rate was incorrectly detected by DVR-scan. The recording device is a SBC (pcduino3 nano). It has an A20 CPU @1Ghz and 1GB RAM, so theoretically, it should be up to the job... not sure what could explain this drop in FPS on 3 clips out of the set of 4.

I am scanning dashcam footage for human movement near my car. When I run dvr-scan -i 2023-08-06_13-54-20*.mp4 -- threshold 200, I get one motion detection and the output is a combination of two different clips. I believe that this is caused by the following discrepancy detected by dvr-scan: _INFO: video_joiner._load_input_videos(): Opened video 2023-08-06_13-54-20-back.mp4 (1280 x 960 at 32.396 FPS). INFO: video_joiner._load_input_videos(): Appending video 2023-08-06_13-54-20-front.mp4 (1280 x 960 at 23.976 FPS). WARNING: video_joiner._load_inputvideos(): Warning: framerate does not match first input. Timecodes may be incorrect.

This somehow causes the program to identify the change from clip 1 to clip 2 as a motion, while it is just the transition between two clips.

Required Information:

Provide a full copy of the command line options you are using, and add --verbosity debug, for example:

root@gaetano-MacBookAir:/mnt/tesladrive# dvr-scan -i /mnt/tesladrive/RecentClips/2023-08-06/2023-08-06_13-54-20*.mp4 --threshold 200 --verbosity debug
INFO: controller._init_dvr_scan(): DVR-Scan v1.5.1
DEBUG: controller._init_dvr_scan(): User config file not found.
DEBUG: controller._init_dvr_scan(): Parsing program options.
INFO: scanner.__init__(): Initializing scan context...
INFO: video_joiner._load_input_videos(): Opened video 2023-08-06_13-54-20-back.mp4 (1280 x 960 at 32.396 FPS).
INFO: video_joiner._load_input_videos(): Appending video 2023-08-06_13-54-20-front.mp4 (1280 x 960 at 23.976 FPS).
WARNING: video_joiner._load_input_videos(): Warning: framerate does not match first input. Timecodes may be incorrect.
INFO: video_joiner._load_input_videos(): Appending video 2023-08-06_13-54-20-left_repeater.mp4 (1280 x 960 at 23.976 FPS).
WARNING: video_joiner._load_input_videos(): Warning: framerate does not match first input. Timecodes may be incorrect.
INFO: video_joiner._load_input_videos(): Appending video 2023-08-06_13-54-20-right_repeater.mp4 (1280 x 960 at 24.000 FPS).
WARNING: video_joiner._load_input_videos(): Warning: framerate does not match first input. Timecodes may be incorrect.
DEBUG: scanner.scan_motion(): Using detector MOG2 with params: kernel_size = 5
INFO: scanner.scan_motion(): Scanning 4 input videos for motion events...
Detected: 1 | Progress:  98%|██ 5976/6116 [02:13<00:03, 44.92 frames/s]
ERROR: scanner.scan_motion(): Failed to decode 4 frame(s) from video, timestamps may be incorrect. Try re-encoding or remuxing video (e.g. ffmpeg -i video.mp4 -c:v copy out.mp4). See https://github.com/Breakthrough/DVR-Scan/issues/62 for details.
INFO: scanner._post_scan_motion(): Processed 5976 frames read in 133.0 secs (avg 44.9 FPS).
INFO: scanner._post_scan_motion(): Detected 1 motion events in input.
INFO: scanner._post_scan_motion(): List of motion events:
-------------------------------------------------------------
|   Event #    |  Start Time  |   Duration   |   End Time   |
-------------------------------------------------------------
|  Event    1  |  00:00:57.7  |  00:00:04.5  |  00:01:02.2  |
-------------------------------------------------------------
INFO: scanner._post_scan_motion(): Comma-separated timecode values:
00:00:57.724,00:01:02.200
INFO: scanner._post_scan_motion(): Motion events written to disk.

Expected Behavior: With a threshold of 200, I was expecting no motion detection in the 4 clips involved.

Computing Environment:

• OS: Linux Mint (5.15.0-78-generic 85-Ubuntu SMP)
• Python Version: Python 3.10.12
• OpenCV Version: 4.8.0

Additional Information: NA

Media [Videos/Images/Screenshots]: 2023-08-06_13-54-20-back.DSME_0001.avi.zip

[Breakthrough]

Whenever you pass multiple files to DVR-Scan, it treats them as a single contiguous video. This allows you to combine multiple videos from a single camera angle, while still handling the case where an event occurs across the boundary of a file.

To achieve what you want, instead for now you'll need to run DVR-Scan once on each disjoint clip. You can automate this by writing a for loop in bash, e.g.:

for f in /mnt/foo/*.mp4
do
  dvr-scan -i $f --threshold 200
done

This has also come up in the past in #135, so this is a frequent enough ask that I think it should be added to DVR-Scan. I'll repurpose this bug report to track adding this as a feature. In the next release, we can add a -n/--no-concatenate flag for use cases similar to this. That should avoid requiring the workaround specified above. The documentation should also be updated to make this behavior more clear.

Although this is working as intended currently, I can see how it can be confusing due to the lack of documentation around how this works. Until that's resolved, I'll keep this tagged as known issue as well. Thanks for filing this!

Edit: Also I want to thank you so much for providing all required information, and a sample - that made triaging/"debugging" this a lot easier. Cheers!

[Solarbytes]

Thank you Brandon,

That did the trick. It was a bit slow on my Linux laptop (old core i5 1.6Ghz) @ 45fps, so I tried with a batch command on Win11 on AMD Ryzen 7530U and that got me 90fps - all of that over a WIFI connection to the storage unit. I then tried to run it on my desktop on a Corei5 9th gen on a local USB 3 connection and I still got 90 fps. Does that sound about right? or can it go faster on a faster CPU?

Also, I don't believe it has anything do do with DVR-scan, but when I got back to my computer, the HDMI display would not come out of power saving mode. There was the occasional SDD drive activity LED flashing but it wasn't consistent with the amount of i/o involved in the process. The bottom line is that I am not sure whether the process had completed or not. Is there any log that I can access to check on that? do I need to use a switch to trigger the creation of a log?

I thought I'd mention it in case you have faced that issue or heard of that issue before.

[Solarbytes]

Never mind about the black screen issue, somehow the driver for my monitor was updated to the "correct" one during a recent Win11 update. I changed it back to the "bad" one and that problem is now solved (my monitor is theoretically only supported up to Win8, but it still works perfectly, so I am using the newer model's driver which supports up to Win10 and voila...).

[Breakthrough]

I've added a workaround for this in the user guide in v1.6, but will defer adding the flag to v1.7.