acon96 / home-llm

A Home Assistant integration & Model to control your smart home using a Local LLM
484 stars 56 forks source link

Unexposed "script" services are fed to the model #119

Closed tannisroot closed 2 months ago

tannisroot commented 2 months ago

Describe the bug
While looking at responses with debug logging on, I've noticed that if I have any script entities exposed in Assistant settings, the integration will list all of my script entities in the Services: instead of just the exposes ones. For example, this is a snippet of the system prompt from debug logging when I have "Turn on my PC" and "Suspend my PC" exposed:

Services: script.flash_entryway_lights(), script.rice_cooker_announcement(), script.urii_pc_turn_on(), script.urii_pc_turn_off(), script.jet_mode(), script.chest_fridge_alert(), script.1705006555075(), script.bedroom_lamp_turn_off(), script.bedroom_humidifier_turn_on(), script.bedroom_humidifier_turn_off(), script.living_room_lamp_turn_off(), script.reset_internet_modem(), script.suspend_my_pc(), script.turn_on_my_pc(), script.reload(), script.turn_on(), script.turn_off(), script.toggle(), script.doorbell(), script.washer_announcement_local(), light.turn_on(), light.turn_off(), light.toggle(), media_player.turn_on(), media_player.turn_off(), media_player.toggle(), media_player.volume_up(), media_player.volume_down(), media_player.media_play_pause(), media_player.media_play(), media_player.media_pause(), media_player.media_stop(), media_player.media_next_track(), media_player.media_previous_track(), media_player.clear_playlist(), media_player.volume_set(), media_player.volume_mute(), media_player.media_seek(), media_player.join(), media_player.select_source(), media_player.select_sound_mode(), media_player.play_media(), media_player.shuffle_set(), media_player.unjoin(), media_player.repeat_set(), climate.turn_on(), climate.turn_off(), climate.toggle(), climate.set_hvac_mode(hvac_mode), climate.set_preset_mode(preset_mode), climate.set_aux_heat(), climate.set_temperature(hvac_mode,temperature), climate.set_humidity(humidity), climate.set_fan_mode(fan_mode), climate.set_swing_mode(), cover.open_cover(), cover.close_cover(), cover.set_cover_position(), cover.stop_cover(), cover.toggle(), cover.open_cover_tilt(), cover.close_cover_tilt(), cover.stop_cover_tilt(), cover.set_cover_tilt_position(), cover.toggle_cover_tilt(), switch.turn_off(), switch.turn_on(), switch.toggle(), button.press(), humidifier.turn_on(), humidifier.turn_off(), humidifier.toggle(), humidifier.set_mode(), humidifier.set_humidity(humidity), vacuum.start(), vacuum.pause(), vacuum.return_to_base(), vacuum.clean_spot(), vacuum.locate(), vacuum.stop(), vacuum.set_fan_speed(), vacuum.send_command(), fan.turn_on(preset_mode), fan.turn_off(), fan.toggle(), fan.increase_speed(), fan.decrease_speed(), fan.oscillate(), fan.set_direction(), fan.set_percentage(), fan.set_preset_mode(preset_mode)
Devices:
cover.bedroom_curtains 'Bedroom Curtains' = open
light.all_lights 'All Lights' = off
light.entryway_lights 'Entryway Lights' = off
light.toilet_lights 'Toilet Lights' = off
light.corridor_lights 'Corridor Lights' = off
light.bathroom_lights 'Bathroom Lights' = off
light.gym_lights 'Gym Lights' = off
light.kitchen_lights 'Kitchen Lights' = off
media_player.living_room_tv 'Living Room TV' = off
button.reboot_shinji 'Reboot Shinji' = unknown
humidifier.bedroom_humidifier_hygrostat 'Bedroom Humidifier' = off;0%
fan.gym_air_purifier 'Gym Air Purifier' = on
switch.bedroom_lamp 'Bedroom Lamp' = off
switch.living_room_lamp 'Living Room Lamp' = off
climate.boiler 'Boiler' = off;23C
climate.air_conditioner 'Air Conditioner' = off;24C
button.intercom_button 'Intercom Button' = 2024-04-18T12:06:28.626232+00:00
light.bedroom_lights 'Bedroom Lights' = off
script.suspend_my_pc 'Suspend My PC' = off
script.turn_on_my_pc 'Turn On My PC' = off
fan.bedroom_air_purifier 'Bedroom Air Purifier' = off
vacuum.roomba 'Roomba' = docked
fan.living_room_air_purifier 'Living Room Air Purifier' = off
media_player.bedroom_tv 'Bedroom TV' = off

While here is one but with those entities not exposed:


Services: light.turn_on(), light.turn_off(), light.toggle(), media_player.turn_on(), media_player.turn_off(), media_player.toggle(), media_player.volume_up(), media_player.volume_down(), media_player.media_play_pause(), media_player.media_play(), media_player.media_pause(), media_player.media_stop(), media_player.media_next_track(), media_player.media_previous_track(), media_player.clear_playlist(), media_player.volume_set(), media_player.volume_mute(), media_player.media_seek(), media_player.join(), media_player.select_source(), media_player.select_sound_mode(), media_player.play_media(), media_player.shuffle_set(), media_player.unjoin(), media_player.repeat_set(), climate.turn_on(), climate.turn_off(), climate.toggle(), climate.set_hvac_mode(hvac_mode), climate.set_preset_mode(preset_mode), climate.set_aux_heat(), climate.set_temperature(hvac_mode,temperature), climate.set_humidity(humidity), climate.set_fan_mode(fan_mode), climate.set_swing_mode(), cover.open_cover(), cover.close_cover(), cover.set_cover_position(), cover.stop_cover(), cover.toggle(), cover.open_cover_tilt(), cover.close_cover_tilt(), cover.stop_cover_tilt(), cover.set_cover_tilt_position(), cover.toggle_cover_tilt(), switch.turn_off(), switch.turn_on(), switch.toggle(), button.press(), humidifier.turn_on(), humidifier.turn_off(), humidifier.toggle(), humidifier.set_mode(), humidifier.set_humidity(humidity), vacuum.start(), vacuum.pause(), vacuum.return_to_base(), vacuum.clean_spot(), vacuum.locate(), vacuum.stop(), vacuum.set_fan_speed(), vacuum.send_command(), fan.turn_on(preset_mode), fan.turn_off(), fan.toggle(), fan.increase_speed(), fan.decrease_speed(), fan.oscillate(), fan.set_direction(), fan.set_percentage(), fan.set_preset_mode(preset_mode)
Devices:
cover.bedroom_curtains 'Bedroom Curtains' = open
light.all_lights 'All Lights' = off
light.entryway_lights 'Entryway Lights' = off
light.toilet_lights 'Toilet Lights' = off
light.corridor_lights 'Corridor Lights' = off
light.bathroom_lights 'Bathroom Lights' = off
light.gym_lights 'Gym Lights' = off
light.kitchen_lights 'Kitchen Lights' = off
media_player.living_room_tv 'Living Room TV' = off
button.reboot_shinji 'Reboot Shinji' = unknown
humidifier.bedroom_humidifier_hygrostat 'Bedroom Humidifier' = off;0%
fan.gym_air_purifier 'Gym Air Purifier' = on
switch.bedroom_lamp 'Bedroom Lamp' = off
switch.living_room_lamp 'Living Room Lamp' = off
climate.boiler 'Boiler' = off;23C
climate.air_conditioner 'Air Conditioner' = off;24C
button.intercom_button 'Intercom Button' = 2024-04-18T12:06:28.626232+00:00
light.bedroom_lights 'Bedroom Lights' = off
fan.bedroom_air_purifier 'Bedroom Air Purifier' = off
vacuum.roomba 'Roomba' = docked
fan.living_room_air_purifier 'Living Room Air Purifier' = off
media_player.bedroom_tv 'Bedroom TV' = off```

**Expected behavior**  
I expect that only services for exposed scripts get exposed and not all of my scripts.
tannisroot commented 2 months ago

Maybe only these services should be exposed and the rest ignored, with scripts being executed through them: script.reload(), script.turn_on(), script.turn_off(), script.toggle()

acon96 commented 2 months ago

Hmm yeah it looks like the current technique of filtering services by the exposed entity types won't work here since a script shows up as a service. I need to figure out a better way of handling this that still gives the user to pick what is exposed to the model without making it a giant 10 page long settings window.

acon96 commented 2 months ago

this should be fixed in v0.2.13. I did what you suggested and made it so that scripts cause script.reload(), script.turn_on(), script.turn_off(), script.toggle() to be exposed.