search

pinokiofactory / cogstudio

86 stars 3 forks source link

Image to Video with 1024x1024 image with frame interpolation on causes tensor size mismatch error (Windows) #8

Open LuckyNES opened 6 hours ago

LuckyNES commented 6 hours ago

I won't attach my test image because it's too weird, but I assume any 1024x1024 image will do the trick. In my case I used a .png image.

resized_img = img.resize((new_width, new_height), Image.ANTIALIAS) Traceback (most recent call last): File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\queueing.py", line 536, in process_events response = await route_utils.call_process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\route_utils.py", line 322, in call_process_api output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\blocks.py", line 1935, in process_api result = await self.call_function( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\blocks.py", line 1520, in call_function prediction = await anyio.to_thread.run_sync( # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\anyio_backends_asyncio.py", line 2357, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\anyio_backends_asyncio.py", line 864, in run result = context.run(func, args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\utils.py", line 826, in wrapper response = f(args, kwargs) ^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\utils.py", line 826, in wrapper response = f(*args, *kwargs) ^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\cogstudio.py", line 703, in generate latents = rife_inference_with_latents(frame_interpolation_model, latents) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife_model.py", line 125, in rife_inference_with_latents frames = ssim_interpolation_rife(model, latent) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife_model.py", line 70, in ssim_interpolation_rife tmp_output = make_inference(model, I0, I1, upscale_amount, 2**exp - 1) if exp else [] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife_model.py", line 26, in make_inference middle = model.inference(I0, I1, upscale_amount) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife\RIFE_HDv3.py", line 54, in inference imgs = torch.cat((img0, img1), 1) ^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 480 but got size 496 for tensor number 1 in the list.

LuckyNES commented 6 hours ago

480x480 image gave the same error.

LuckyNES commented 4 hours ago

I think that interpolation just does not work at least on Windows.

fleagle2001 commented 2 hours ago

I'm on Windows, and I was getting this error when I didn't use Super Resolution (the upscaling option next to interpolation). So far I haven't had an issue with having interpolation on. The error I receive when not using the upscaling method is identical to yours. When you get the error, do you have the Super Resolution option checked?