jbilcke-hf / clapper

Clapper.app, a video synthesizer and sequencer designed for the age of AI cinema
https://clapper.app
GNU General Public License v3.0
2.08k stars 195 forks source link

Use a LLM to infer the entity description prompt #77

Open jbilcke-hf opened 3 months ago

jbilcke-hf commented 3 months ago

Context

Currently entity definition is done in a very crude way, based on the name, age and gender of character (and those are actually guessed, they are not reliable)

same for non-character entities (eg. locations

While we offer to users the possibility to edit the entity settings, this is a slow process when you have multiple characters, and some users only want to focus on the fun part of creating and visualizing a story, not doing the data entry (it is a movie assistant job)

Solution

Use a LLM to infer the characteristics, by using the scene and/or synopsis and/or the full screenplay as a context.