search

Vikit-ai / sdk

Vikit.ai SDK let you develop easily video generators leveraging generative AI and other AI models.
https://vikit.ai
Apache License 2.0
51 stars 7 forks source link

Imposing video style #115

Open leclem opened 3 months ago

leclem commented 3 months ago

Problem: When creating a composite, video style can vary from one video to another. For example, the first video could show a very realistic human, and the second a cartoonish-style human. This can hurt the consistency and coherence of the final content.

Imposing video style I want to be able to impose or avoid certain styles. E.g., generating all videos in cartoon style or all in realistic style

My proposed solution for Stability AI, we can set up the style_preset and negative prompting, for the text2image part, we can enforce a non-cartoonish style. From their API documentation:

Enum: 3d-model analog-film anime cinematic comic-book digital-art enhance fantasy-art isometric line-art low-poly modeling-compound neon-punk origami photographic pixel-art tile-texture
Guides the image model towards a particular style.

As for Haiper, we don't have an explicit style preset, but we can get inspired by their web API, inject styles directly in the prompt, or try to add cartoonish style in the negative prompt, to be tested! For VideoCrafter I don't have an immediately applicable solution, but we can follow the same idea using negative prompting (once it is implemented)

jeffmac-aix commented 3 months ago

Need a design that factors in the target model along with a buildsettings prompt's negative prompt