Image to Video with 1024x1024 image with frame interpolation on causes tensor size mismatch error (Windows)

[LuckyNES]

I won't attach my test image because it's too weird, but I assume any 1024x1024 image will do the trick. In my case I used a .png image.

resized_img = img.resize((new_width, new_height), Image.ANTIALIAS) Traceback (most recent call last): File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\queueing.py", line 536, in process_events response = await route_utils.call_process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\route_utils.py", line 322, in call_process_api output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\blocks.py", line 1935, in process_api result = await self.call_function( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\blocks.py", line 1520, in call_function prediction = await anyio.to_thread.run_sync( # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\anyio_backends_asyncio.py", line 2357, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\anyio_backends_asyncio.py", line 864, in run result = context.run(func, args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\utils.py", line 826, in wrapper response = f(args, kwargs) ^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\gradio\utils.py", line 826, in wrapper response = f(*args, *kwargs) ^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\cogstudio.py", line 703, in generate latents = rife_inference_with_latents(frame_interpolation_model, latents) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife_model.py", line 125, in rife_inference_with_latents frames = ssim_interpolation_rife(model, latent) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo.venv\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife_model.py", line 70, in ssim_interpolation_rife tmp_output = make_inference(model, I0, I1, upscale_amount, 2**exp - 1) if exp else [] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife_model.py", line 26, in make_inference middle = model.inference(I0, I1, upscale_amount) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\GitRepos\CogVideo\inference\gradio_composite_demo\rife\RIFE_HDv3.py", line 54, in inference imgs = torch.cat((img0, img1), 1) ^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 480 but got size 496 for tensor number 1 in the list.

[LuckyNES]

480x480 image gave the same error.

[LuckyNES]

I think that interpolation just does not work at least on Windows.

[fleagle2001]

I'm on Windows, and I was getting this error when I didn't use Super Resolution (the upscaling option next to interpolation). So far I haven't had an issue with having interpolation on. The error I receive when not using the upscaling method is identical to yours. When you get the error, do you have the Super Resolution option checked?

[LuckyNES]

@fleagle2001 I do not have that Super Resolution option checked when I do interpolation. Good to know. At least it's a way to make it work. But I hope for a bug fix or at least a warning message.

[stealthseal2]

Getting the same error as well with frame interpolation on: "RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 480 but got size 496 for tensor number 1 in the list."

[shutarojp]

try resize or add black bar to match their video 720x480, I find they have only this size for us, even you add 1920x1080 and the output is only 720x480 and they added black bar to empty area

[LuckyNES]

This program should handle it automatically, shouldn't it?

[shutarojp]

This program should handle it automatically, shouldn't it?

I don't think so