Interpolate from External Sketches

[warudokiruhya]

How do I do the Interpolate from External Sketches? There is no option to add more than 1 image.

[toyxyz]

My slightly modified version allows you to upload keyframes! https://github.com/toyxyz/Paints-UNDO

[warudokiruhya]

I Installed that now, but where do I add the sketch? I want to add the finished one and the sketch to then get the process like you showed. When I upload it to the keyframe it is not being used and just gets erased and fully replaced with the new generated ones. If you mean that I upload it to 3_2, that means I need to generate the video first, and then generate the video again at 3_2 for it to work? I get an error at every time, before generating the video, and after it too.

Traceback (most recent call last):
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\queueing.py", line 528, in process_events
    response = await route_utils.call_process_api(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\route_utils.py", line 270, in call_process_api
    output = await app.get_blocks().process_api(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\blocks.py", line 1908, in process_api
    result = await self.call_function(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\blocks.py", line 1485, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\_backends\_asyncio.py", line 2177, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\_backends\_asyncio.py", line 859, in run
    result = context.run(func, *args)
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\utils.py", line 808, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\git\Paints-UNDO-main_alt\gradio_app.py", line 229, in process_video
    video = torch.cat(result_frames, dim=2)
RuntimeError: torch.cat(): expected a non-empty list of Tensors
Traceback (most recent call last):
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\queueing.py", line 528, in process_events
    response = await route_utils.call_process_api(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\route_utils.py", line 270, in call_process_api
    output = await app.get_blocks().process_api(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\blocks.py", line 1908, in process_api
    result = await self.call_function(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\blocks.py", line 1485, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\_backends\_asyncio.py", line 2177, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\_backends\_asyncio.py", line 859, in run
    result = context.run(func, *args)
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\utils.py", line 808, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\git\Paints-UNDO-main_alt\gradio_app.py", line 229, in process_video
    video = torch.cat(result_frames, dim=2)
RuntimeError: torch.cat(): expected a non-empty list of Tensors
Traceback (most recent call last):
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\queueing.py", line 528, in process_events
    response = await route_utils.call_process_api(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\route_utils.py", line 270, in call_process_api
    output = await app.get_blocks().process_api(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\blocks.py", line 1908, in process_api
    result = await self.call_function(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\blocks.py", line 1485, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\_backends\_asyncio.py", line 2177, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\anyio\_backends\_asyncio.py", line 859, in run
    result = context.run(func, *args)
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\gradio\utils.py", line 808, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Ayu\miniconda3\envs\paints_undo\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\git\Paints-UNDO-main_alt\gradio_app.py", line 229, in process_video
    video = torch.cat(result_frames, dim=2)
RuntimeError: torch.cat(): expected a non-empty list of Tensors```

[warudokiruhya]

Please tell me how I can use it and feed it the sketch and the finished one, because the usual way of creating is not usable for me, since the first half of the video is always a fully different looking image.

[warudokiruhya]

I found out that it works when I put in 2 images, but the lineart one never is shown in the video. How many keyframes do I need for it to look like in the example?

[toyxyz]

Often, the first frame is too short. Put two frames of line art, or add a blank white image in front of it.

https://github.com/user-attachments/assets/d4cb66b7-af3b-45f5-afc8-02b5e3644502

[warudokiruhya]

Thank you so much, it worked with this white paper trick. So I assume if I add more keyframes, the result gets even better? I had the idea of adding some keyframes from step 2 but there seems to be no way of downloading them.

[toyxyz]

Thank you so much, it worked with this white paper trick. So I assume if I add more keyframes, the result gets even better? I had the idea of adding some keyframes from step 2 but there seems to be no way of downloading them.

If you add more keyframes, you will get a more natural animation.

[warudokiruhya]

Thank you. Is there a way of exporting keyframes from step 2 to add them at 3_2? Not sure how else to get more keyframes.

[toyxyz]

Thank you. Is there a way of exporting keyframes from step 2 to add them at 3_2? Not sure how else to get more keyframes.

To download a keyframe image of a specific step, press the Download button.

[warudokiruhya]

That button only downloads a text file somehow and inside is strange stuff like

RIFF¸0 WEBPVP8 ¬0 °(

€>m6–H¤#"!#Sð€ ‰inöŠrÊ Ÿ°1#OI~Åzêú?ò3·‹µ_ç¿ŸóýÙíþ ¹+׋q=
p/üO5…ÿCìÁÛø¿úÞÀ¿Ï?ÒzÊ÷þý§~XáUUUSý­ÔÇÖiNØ=C1N<°®j^´9ø@d—i¬Nîîîîîîîîîî —iöõñ"ù´ m¨Å°/ˆP‰ý{Û»òÄÝÝÝÝÝò½™™|TúÅ/€Žåäó{lÑKVŒ|#Ç&[C¬ûRˆð}0A£öÕ©¸Àž·%iÒ4gÍ£30uŠ­sM 8Ù«<!äÆùà8ªË@+›+á/Ž´‹Sr jø±5¾JÓ¤•´fa”¶™—ÿ¨F1‹Áœ–DØ´ñ²†7‰ÉÖŽ“]'Æï×âNêQ#/¿´µƒ7ÉZtshÌ¥dM’!°Òéƒ.˜£µÉ¸ýŠ÷Ä(ü2¹zoh JWÚÖ‡gHŹ<ßìø¼ÒM£í¬Œ}IdPu(:aÝÝÝÞ
iÒ,VZÙ¼iþ»ðŠRBeYkn{ ,šèµ|ÏnMJY¶©µP5AJ€"H ž<°ˆ"+’†>y2Ôsùö§T‘£33>m…‚c¥u×óáퟣ›…W%&»PéQ+€Åv{ÝJ†».V>%N\ä’8Äó²gZ7õüœµÖuh×wÒMÔûd±;»¼Ó¨üŠ–˜ªAŽ ÓWØgäª)) Wÿ­ï$ž`„6EÿàÅ~ÊÄtíüjX1á¢t)·mP¦)! §!j\t¿­ò8¼ñÊ HCr°Î³33333e”‰¯Ä¡Nߤ-x·b½K,>y/ÛÙ0~}Œ°
ËUõ©S3f›Ï2}Sæa°¿e#B}mþä

[toyxyz]

That button only downloads a text file somehow and inside is strange stuff like

RIFF¸0 WEBPVP8 ¬0 °(��� �€�>m6–H¤#"!#S�ð€ ‰inöŠ�rÊ Ÿ°1#OI~Åzêú?ò�3·‹µ_ç¿ŸóýÙ�íþ ¹+׋q=�p/üO5�…ÿCì�ÁÛø¿úÞÀ¿Ï?ÒzÊ÷þý§~XáUUUSý­ÔÇÖiNØ=C1N�<°®j�^´9ø@d�—i¬Nîîîîîîîîîî —�iöõñ"ù´ m¨Å°/ˆP‰ý{Û»òÄÝÝÝÝÝò½�™™|TúÅ/€Ž�åäó���{lÑKVŒ|#Ç&[C¬ûR�ˆð}�0A£öÕ©¸Àž·%iÒ4gÍ£30�uŠ­sM 8Ù«�<�!äÆùà��8ªË@+›+á�/Ž�´‹Sr j_ø±5¾JÓ¤•´fa”¶™—ÿ¨F�1‹Áœ–DØ´�ñ��²†7‰ÉÖ_Ž“]'Æï�×âNêQ�#/¿´µ��ƒ7ÉZt�shÌ�¥d��M’!°Òéƒ.�˜£µÉ¸ýŠ÷Ä(ü2¹zoh JWÚÖ‡�gHŹ�<ßìø¼ÒM£í¬Œ}IdPu(:�aÝÝÝÞ�iÒ,V�ZÙ¼iþ»ðŠRBe�Ykn{ ,�šèµ��|ÏnMJ�Y¶©µP5AJ€"H ž<°ˆ"+’†�>y2Ôsùö§T�‘£33>m���…‚�c¥u×óáí�Ÿ£›…W%&»PéQ+€Å�v{ÝJ†».V>%N\ä’8Äó²gZ7õ�üœµ��Öu�h×wÒ�MÔûd±;»¼�Ó¨üŠ–��˜�ªAŽ ÓWØgäª)) Wÿ­ï$ž`„6EÿàÅ~ÊÄtíüjX��1á¢t)·mP¦)! §!j\t¿­ò8¼ñÊ H�Cr°Î³3333�3�e”‰¯Ä�¡Nߤ-x·�b½K,>y/ÛÙ0~}Œ°�ËU�õ©S3f›Ï2}Sæa°¿e#B}mþä

That's an image file in webp format. Try opening it with an image viewer. https://en.bandisoft.com/bandiview/

[warudokiruhya]

Its really a .txt file for me but I discovered I can rename the extension to png and then it shows.

[warudokiruhya]

Any recommended settings for the CFG scale and FPS? I wish there was a setting to make the video longer. Its always 8 seconds.

[toyxyz]

Any recommended settings for the CFG scale and FPS? I wish there was a setting to make the video longer. Its always 8 seconds.

If you lower the fps, the video will play slower.