search

jekalmin / extended_openai_conversation

Home Assistant custom component of conversation agent. It uses OpenAI to control your devices.
834 stars 108 forks source link

Function spec examples for camera and image query #175

Closed danielp370 closed 3 months ago

danielp370 commented 3 months ago

This PR provides an example functions for:

  1. generate a camera snapshot image.
  2. use the query image service call to analyze the image and return a text description
  3. A combined function that does 1 and 2 above.

All three functions tested.

The integration works well with 1 and 2 defined, and the LLM figures out how to combined them, so option 3 is only for those that want really want to combined them.

I recommend keeping filename exposed so that the LLM can manage it, and potentially delete it or overwrite it.