search

jekalmin / extended_openai_conversation

Home Assistant custom component of conversation agent. It uses OpenAI to control your devices.
834 stars 108 forks source link

FR: Ability to query image from media directory #140

Open saya6k opened 4 months ago

saya6k commented 4 months ago

Currently, the extended_openai_conversation.query_image function can only retrieve images from publicly accessible URLs, specifically from the www directory, potentially leading to unnecessary public exposure of these images.

For a better fit within the Home Assistant ecosystem and as a better security practice, it would be advantageous if this function could also access media from the media_dir, offering a more seamless integration.

Integrations like the Gallery Card already support accessing the media directory. Alternatively, methods such as using the /www/tmp directory could be employed.

Adding the capability to query images from the media directory would enhance functionality and security.

jekalmin commented 4 months ago

I totally agree that uploading image without exposing to public is needed. I will look into it.

TBTHEDBA commented 4 months ago

The new google integration can do this if needed in the mean time. google_generative_ai_conversation.generate_content

jekalmin commented 4 months ago

I just released 1.0.4-beta1.

Please try like below and give a feedback.

AbortRetry commented 3 months ago

Working for me thanks. Taking a snapshot of the doorbell cam and sending it out and receiving a response. Now just to work out using the response data hehe