Send multiple images using the same prompt

PatrikPatrik2 commented 2 weeks ago

I am trying to get image analysis automation of my surveillance cameras - I have 3 cameras in the same area, and I would like to send all three images at once (with one prompt) to the image analysis - and maybe even the the previous three images such that the LLM might be able to analyze motion / difference.

Would it be possible to include multiple images in a single call?

mediacutlet commented 2 weeks ago

@PatrikPatrik2 I have had success by simply putting each image on a separate line.

sequence:
  - action: llmvision.image_analyzer
    metadata: {}
    data:
      provider: OpenAI
      model: gpt-4o
      include_filename: false
      detail: high
      max_tokens: 300
      temperature: 0.7
      image_file: |-
        /config/www/weather_radar/nam/nam-hires_us-ne_001_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_002_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_003_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_004_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_005_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_006_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_007_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_008_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_009_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_010_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_011_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_012_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_013_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_014_sim_radar_1km.gif
        /config/www/weather_radar/nam/nam-hires_us-ne_015_sim_radar_1km.gif
      message: >-

PatrikPatrik2 commented 2 weeks ago

Perfect, thanks!!!!

valentinfrlch commented 2 weeks ago

Additionally, you can use templates to list all currently recording cameras:

"{{ states.camera|selectattr("state","eq","recording")|map(attribute='entity_id')|list}}"

valentinfrlch / ha-llmvision

Send multiple images using the same prompt #60