twelvelabs-io / tl-jockey

Jockey is a conversational video agent.
51 stars 13 forks source link

Explore adding dynamic prompting for instructor #47

Open TravisCouture opened 4 months ago

TravisCouture commented 4 months ago

Currently, the instructor always leverages the same prompt for each worker. This likely causes confusion and unintended behavior due to irrelevant noise in the prompt for the active worker (video-text-generation instructors are mostly useless for the video-search worker for example). It may be possible to use the next_worker state variable to dynamically load prompts at execution time so that the instructor can simultaneously receive more details in the system prompt and provide clearer and fuller instructions to the target worker.