memory problem

[amandalucs]

Hi, congratulations o your work! :-) So, I'm trying to use katna on a windows 10 machine with 16gb ram, python 3.8 and katna 0.9.0. I am testing using a 2,6gb video and my memory keeps reaching 100% and the script eventually crashes. I also tried setting the Katna.config.FrameExtractor.max_frames_in_chunk from 2500 to 100 and Katna.Video.n_processes to 1, but still got the same problem. Any idea of how to solve this?

This is a snippet of the code I'm running:

if __name__ == "__main__":

  # initialize video module
  vd = Video()
  print('vd.n_processes:')
  print(vd.n_processes)

  # number of images to be returned
  no_of_frames_to_returned = 12
  diskwriter = KeyFrameDiskWriter(location=out_path)

  # extract keyframes and process data with diskwriter
  vd.extract_video_keyframes(
       no_of_frames=no_of_frames_to_returned, file_path=video_file_path,
       writer=diskwriter
  )```

[amandalucs]

The last execution generated the following output:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\amandalucas\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Users\amandalucas\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\amandalucas\AppData\Local\Programs\Python\Python38\lib\multiprocessing\pool.py", line 576, in _handle_results
    task = get()
  File "C:\Users\amandalucas\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "C:\Users\amandalucas\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py", line 318, in _recv_bytes
    return self._get_more_data(ov, maxsize)
  File "C:\Users\amandalucas\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py", line 337, in _get_more_data
    assert left > 0
AssertionError

[amandalucs]

I tried your sample video and it works just fine. I guess the problem is either the size or the compression of the video I'm using (mpg).

[thealokkr]

Hi Amanda, Most probably, it's the file size causing the issue. The number of frames for a 2 GB file would be extremely high. To isolate the issue, Can you try with a smaller sized MPG file? You can use https://www.ffmpeg.org/ to split your current file

[mayank10j]

Released in Katna v0.9.1 support for processing of Video of size greater than 20 minutes without memory issue.

[mayank10j]

Please let me know if you are able to use it on bigger video and reopen if their is still issue.