search

Doubiiu / ToonCrafter

[SIGGRAPH Asia 2024, Journal Track] ToonCrafter: Generative Cartoon Interpolation
https://doubiiu.github.io/projects/ToonCrafter/
Apache License 2.0
5.32k stars 440 forks source link

Suggestion: Make a lite version that only cleans up existing human made animations #37

Open nehochupechatat opened 5 months ago

nehochupechatat commented 5 months ago

I'm not sure if this would work, but wouldn't the cleanup (reference-based sketch colorization) part of the AI be easy to isolate? It'd also be lighter on system resources probably.

Many human animators like to rough animate but cleanup is tedious. The clean-up examples also look the best out of them all.

Inputting PNG sequences would be nice too.

zafirexd commented 5 months ago

Use Coarse Realistic Lineart Controlnet Preprocessor image https://github.com/lllyasviel/ControlNet-v1-1-nightly image https://huggingface.co/spaces/hysts/ControlNet-v1-1

nehochupechatat commented 5 months ago

This is different because ToonCrafter is for 2D, not uncanny ""photorealistic"" art.

zafirexd commented 5 months ago

Just try it here https://huggingface.co/spaces/hysts/ControlNet-v1-1 image Its not perfect because the artists who should be making this tools.....hates ai, but you can train your own custom preprocessor for the specific task you want, you wont find anything "Production Quality" because, again, 2d animators hates ai, good luck training a rough animation cleaner, in the illyasviel/ControlNet repo you can find how to do it. image image image High resolution imgs makes better results. Using Rough animation as final colorized animation is also an art style. https://youtu.be/g2T6lJaodlE