Scanner - Out of Memory and long estimate after latest changes

[dege13]

It looks like some of the settings that changed in the last code change caused a new issue. Memory usage of python.exe is over 28GB with a 5.3GB MKV seen below with an estimate of over 13 hours.

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::o     o  o      8                 ooooo               8        ::
::8     8         8                   8                 8        ::
::8     8 o8 .oPYo8 .oPYo. .oPYo.     8   .oPYo. .oPYo. 8 .oPYo. ::
::`b   d'  8 8    8 8oooo8 8    8     8   8    8 8    8 8 Yb..   ::
:: `b d'   8 8    8 8.     8    8     8   8    8 8    8 8   'Yb. ::
::  `8'    8 `YooP' `Yooo' `YooP'     8   `YooP' `YooP' 8 `YooP' ::
:::::..::::..:.....::.....::.....:::::..:::.....::.....:..:.....:::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
: ___  __              ___     __   __                  ___  __  ::
:|__  |__)  /\   |\/| |__     /__` /  `  /\  |\ | |\ | |__  |__) ::
:|    |  \ /~~\  |  | |___    .__/ \__, /~~\ | \| | \| |___ |  \ ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
2023-04-17 16-32-04 - BASF chrome extra quality - Jane - Raiders of the lost ark, European Vacation.mkv
on 2662: Exception in thread Thread-3 (update):
Traceback (most recent call last):
  File "C:\Users\dege1\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
on 2664:     self.run()
on 2664:   File "C:\Users\dege1\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
on 2664:   File "C:\Users\dege1\AppData\Local\Programs\Python\Python310\lib\site-packages\imutils\video\filevideostream.py", line 47, in update
    (grabbed, frame) = self.stream.read()
on 2664: cv2.error: OpenCV(4.7.0) D:\a\opencv-python\opencv-python\opencv\modules\core\src\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 1036800 bytes in function 'cv::OutOfMemoryError'
|▎                                       | ▆▄▂ 4712/639567 [1%] in 5:48 (~13:02:02, 13.5/s)

[MoeFwacky]

I've added a limit to the Queue that should take care of this error. Please reopen if this isn't the case, but I'm running it on a large file now and it appears stable.

[dege13]

The memory issue looks to be fixed. However, the estimated time is still much longer than it used to be. An 8 hour file used to take under 15 minutes. Now the estimate is over 2 hours.

2023-04-17 16-32-04 - BASF chrome extra quality - Jane - Raiders of the lost ark, European Vacation.mkv
|▏                                       | ▇▅▃ 2220/639567 [0%] in 34s (~2:41:35, 65.7/s)

I will retest, but I am pretty sure the new changes are converting the file causing it to take longer. I assume this is to fix issues with badly encoded files where there were frame count issues causing the old hang. I am not sure the trade off is worth, and hope there is another alternative. Perhaps your original OBS recording settings are something I should be using as well to prevent this re-encode.

It looks like GitHub does not give me an option to reopen closed issues.

[dege13]

Looks good after rolling back the changes. previous version: 2023-04-17 16-32-04 - BASF chrome extra quality - Jane - Raiders of the lost ark, European Vacation.mkv |████████████████████████████████████████| 639567/639567 [100%] in 2:46:05.4 (64.18/s) [INFO] elasped time: 9965.42 [INFO] approx. FPS: 64.18

Current version: 2023-04-17 16-32-04 - BASF chrome extra quality - Jane - Raiders of the lost ark, European Vacation.mkv on 639566: axis 0 is out of bounds for array of dimension 0 on 639567: [INFO] elasped time: 539.51 on 639567: [INFO] approx. FPS: 1185.46 |████████████████████████████████████████| 639567/639567 [100%] in 8:59.4 (1185.69/s)

I hope "axis 0 is out of bounds for array of dimension 0" is not a sign of any issues. I will continue testing more files and put this file into the splitter and see what happens.

[dege13]

Unfortunately the original issue of hanging at 100% still exists because of rolling back the fix.

2022-11-26 00-09-42 - JVC #42 - Devil's Dorway, Little Big Man, River of No Return.mp4_trim.mkv
on 427645: axis 0 is out of bounds for array of dimension 0
|████████████████████████████████████████| ▅▃▁ 427646/427706 [100%] in 12:05 (~0s, 590.0/s)

[MoeFwacky]

I've reopened the hanging at 100% issue, but since this one isn't happening anymore, I'm closing it (though if it comes back I'll reopen).