jekalmin / extended_openai_conversation

Home Assistant custom component of conversation agent. It uses OpenAI to control your devices.
916 stars 129 forks source link

Getting variable from previous step inside a sequence #119

Closed gterras closed 8 months ago

gterras commented 8 months ago

Hi and thank you for this great lib,

I'm trying to get the Spotify plus integration to work, by using the search service provided. With a query like "Play some Pantera":

- spec:
    name: search_music
    description: Use this function to search music
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The query
      required:
      - query
  function:
    type: composite
    sequence:
    - type: script
      sequence:
      - service: spotify_plus.spotify_search
        data:
          search_type: Artist Profile
          search_term: "{{ query }}"
    - type: template
      value_template: >-
        media_id,media_name
        {% set results = state_attr('sensor.spotify_search', 'search_results') %}
        {%- for track in results.tracks -%}
            {{ track.uri }},{{ track.name }} 
        {%- endfor %}
    - type: script
      sequence:
      - service: media_player.play_media
        target:
          entity_id: media_player.spotify
        data:
          media_content_id: '{{ media_id }}' // how to get the media_id here? 
          media_content_type: music

I was under the naive assumption that outputting the result of the search in value_template would be enough for a value to be automatically picked by GPT but it doesn't look like it's the case, even with a simple output like {{ results.tracks[0].uri }}.

Is value_template actually the way to go? Note that hardcoding media_content_id will work.

Thanks for your help.

jekalmin commented 8 months ago

You can achieve this in two ways. (Since I haven't tested, there might be typos)

1. search and play music at once (like you did)

In composite function, you can use response_variable to pass variable to next sequence.

- spec:
    name: search_music
    description: Use this function to search music
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The query
      required:
      - query
  function:
    type: composite
    sequence:
    - type: script
      sequence:
      - service: spotify_plus.spotify_search
        data:
          search_type: Artist Profile
          search_term: "{{ query }}"
    - type: template
      value_template: >-
        {% set results = state_attr('sensor.spotify_search', 'search_results') %}
        {{- results.tracks[0].uri -}}
      response_variable: "media_id" // pass whatever returned from "value_template" as "media_id" to next sequence.
    - type: script
      sequence:
      - service: media_player.play_media
        target:
          entity_id: media_player.spotify
        data:
          media_content_id: '{{ media_id }}' // You can use "media_id" because `response_variable` from previous template is "media_id"
          media_content_type: music

2. search and play function separately

You can register two functions, search_music and play_music. Hopefully LLM calls search_music first to get media_id, and then play_music using media_id.

- spec:
    name: search_music
    description: Search music
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The query
      required:
      - query
  function:
    type: composite
    sequence:
    - type: script
      sequence:
      - service: spotify_plus.spotify_search
        data:
          search_type: Artist Profile
          search_term: "{{ query }}"
    - type: template
      value_template: >-
        media_id,media_name
        {% set results = state_attr('sensor.spotify_search', 'search_results') %}
        {%- for track in results.tracks -%}
            {{ track.uri }},{{ track.name }} 
        {%- endfor %}
- spec:
    name: play_music
    description: Play music
    parameters:
      type: object
      properties:
        media_id:
          type: string
          description: The media_id
      required:
      - media_id
  function:
    - type: script
      sequence:
      - service: media_player.play_media
        target:
          entity_id: media_player.spotify
        data:
          media_content_id: '{{ media_id }}'
          media_content_type: music
gterras commented 8 months ago

Now this is top-notch support if I've ever seen one, thank you :heart:

Hopefully LLM calls search_music first to get media_id, and then play_music using media_id

Wow I wasn't aware it was able to compose with functions, might be a little risky but really good to know!