uezo / ChatdollKit

ChatdollKit enables you to make your 3D model into a chatbot
Apache License 2.0
684 stars 73 forks source link

Support autonomous vision input for Claude✹ #304

Closed uezo closed 3 weeks ago

uezo commented 3 weeks ago

Implement functionality for Claude to autonomously determine when to capture images (e.g. from a camera) based on user requests. Enhanced the agent's ability to handle multimodal inputs for improved user interaction.

Also support Function Calling. But currently, only the first content in the response is processed. Ensure that the prompt controls for content to include only tool_use.