zzh-tech / InterpAny-Clearer

[ECCV2024 Oral] Clearer anytime frame interpolation & Manipulated interpolation of anything
https://zzh-tech.github.io/InterpAny-Clearer/
MIT License
207 stars 12 forks source link

Memory error and performance issue #7

Closed VicMahoney closed 9 months ago

VicMahoney commented 9 months ago

@zzh-tech Hello again! I have rtx 3060 ti 8gb. When I tried to do a test of a 10 second 4K video, I got this error: numpy.core._exceptions.MemoryError: Unable to allocate 23.9 MiB for an array with shape (2176, 3840, 3) and data type uint8

Also a minute of full hd video took almost 30 minutes on default settings (demo with model RIFE --variant DR) and got the same memory error . numpy.core._exceptions.MemoryError: Unable to allocate 2.70 MiB for an array with shape (736, 1280, 3) and data type uint8 And in general, the question is, how many models are there to try? How to change models on the command line?

After all this I got an error Traceback (most recent call last): File "A:\InterpAny\models\DI-RIFE\inference_video_plus_sdi_recur.py", line 10, in import cv2 ModuleNotFoundError: No module named 'cv2'

Upd i understood how to solve the last problem - with every new start terminal,we need to run "conda activate InterpAny" and there is no error in 'cv2'

zzh-tech commented 9 months ago

I have rtx 3060 ti 8gb.

8gb is too small for 4K resolution.

how many models are there to try? How to change models on the command line?

There are four different models, including RIFE, IFRNet, AMT-S, and EMA-VFI.

How to change models on the command line?

Please set [MODEL_NAME] to a specific model name, e.g., --model AMT-S.

VicMahoney commented 9 months ago

@zzh-tech With flowframes rife 4.6 4k going nice. Well, let’s say it’s clear with 4k, with 1080p it shows the same error, what minimum memory does the program need?

zzh-tech commented 9 months ago

Using [D] instead of [D,R] can save the required memory.

what minimum memory does the program need?

It depends on the model. Using [D] should cost the same amount of memory as each original model. However, we did not count the memory cost for each model.

VicMahoney commented 9 months ago

@zzh-tech where is it written about the difference between D and R?what is that? Upd1 I tried the option "D" with rife model and there are same error with memory

upd 2 determined that the problem is not in the video card memory, but in the RAM - 32 gb is not enough upd3 we need more than 32 gb, i I had to enable the swap file and now everything works

zzh-tech commented 9 months ago

where is it written about the difference between D and R?what is that?

[D] distance indexing [R] iterative reference-based estimation

See the project page or paper for details.

upd3 we need more than 32 gb, i I had to enable the swap file and now everything works

Glad to hear this :-)