feat: add daily_chat_tools_vision_bot and test_vision_sink_processor - Githubissues

ai-bot-pro / achatbot

An open source chat bot architecture for voice/vision (and multimodal) assistants, local and remote to run; if u run achatbot by yourself, u can learn more, star and fork to contribute~

BSD 3-Clause "New" or "Revised" License

15 stars 2 forks source link

feat: add daily_chat_tools_vision_bot and test_vision_sink_processor #65

Closed weedge closed 1 month ago

weedge commented 1 month ago

feat:

add daily_chat_tools_vision_bot and test_vision_sink_processor
ai_conf add tools config
use latest transformers v4.45.0 https://github.com/huggingface/transformers/releases/tag/v4.45.0
add ImageCaptureProcessor
add record_llm_usage_tokens in BaseOpenAILLMProcessor and OpenAIGroqLLMProcessor
add llm,tts token usage metrics

fix:

VisionImageFrameAggregator pass VisionImageRawFrame
empty describe image question
change capture_participant_video framerate when push UserImageRequestFrame and use apipeline.processors.logger FrameLogger to debug
change AudioCameraOutputProcessor sink DataFrame
move openai llm context to openai_llm_context.py