search

acon96 / home-llm

A Home Assistant integration & Model to control your smart home using a Local LLM
664 stars 68 forks source link

Triple backticks at the end of every response #80

Closed tibmeister closed 8 months ago

tibmeister commented 8 months ago

Describe the bug Every response has three backticks (```) at the end of the response.

Expected behavior No extra characters at the end of the response

Logs Example of output is:

The temperature in Megans Room is 70.5. ```

Raw data is:

sensor.temp02_air_temp 'Megans Room Temperature' = 70.5

Prompt is:

You are 'Al', a helpful AI Assistant that controls the devices in a house. Complete the following task ask instructed with the information provided only. All temperature units are in fahrenheit.

Services: {{ services }} Devices: {{ devices }}

Using Zephyr as the Prompt format. Backend model is Home-3B.q5_k_m.gguf on LocalAI engine running in docker container on remote (TrueNAS) with nVidia Quadro M2000.

Debug log from localai: ### Response: 8:32PM DBG Prompt (after templating): Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction: You are 'Al', a helpful AI Assistant that controls the devices in a house. Complete the following task ask instructed with the information provided only. All temperature units are in fahrenheit.

Services: media_player.turn_on(), media_player.turn_off(), media_player.toggle(), media_player.volume_up(), media_player.volume_down(), media_player.media_play_pause(), media_player.media_play(), media_player.media_pause(), media_player.media_stop(), media_player.media_next_track(), media_player.media_previous_track(), media_player.clear_playlist(), media_player.volume_set(volume_level), media_player.volume_mute(), media_player.media_seek(), media_player.join(), media_player.select_source(), media_player.select_sound_mode(), media_player.play_media(), media_player.shuffle_set(), media_player.unjoin(), media_player.repeat_set(), switch.turn_off(), switch.turn_on(), switch.toggle(), input_select.reload(), input_select.select_first(), input_select.select_last(), input_select.select_next(), input_select.select_option(), input_select.select_previous(), input_select.set_options(), scene.reload(), scene.apply(), scene.create(), scene.delete(), scene.turn_on(), script.1705895141158(), script.reload(), script.turn_on(), script.turn_off(), script.toggle(), light.turn_on(brightness,rgb_color), light.turn_off(), light.toggle(brightness,rgb_color) Devices: script.1705895141158 'Play Music' = off input_select.living_room_state 'Living Room Light State' = all-off sensor.groot_system_temperature 'Groot System Temperature' = 104 sensor.living_room_temperature 'Living Room Temperature' = 71.5 sensor.living_room_humidity 'Living Room Humidity' = 50 switch.groot_service_avahi_status 'Groot Service avahi status' = on sensor.pirateweather_temperature 'PirateWeather Temperature' = 65 sensor.pirateweather_apparent_temperature 'PirateWeather Apparent Temperature' = 72 sensor.pirateweather_dew_point 'PirateWeather Dew Point' = 60 sensor.pirateweather_humidity 'PirateWeather Humidity' = 80 light.owl_lamp 'Owl Lamp' = on;27% scene.living_room_good_night 'Living Room Good Night' = 2024-03-04T05:06:42.817971+00:00 sensor.idrac_power_poweredge_r440_cpu1_temp 'iDRAC Power PowerEdge R440 CPU1 Temp' = unknown sensor.idrac_power_poweredge_r440_system_board_inlet_temp 'iDRAC Power PowerEdge R440 System Board Inlet Temp' = unknown sensor.idrac_power_poweredge_r440_system_board_exhaust_temp 'iDRAC Power PowerEdge R440 System Board Exhaust Temp' = unknown sensor.idrac_power_poweredge_r440_cpu1_temp_2 'iDRAC Power PowerEdge R440 CPU1 Temp' = unknown sensor.idrac_power_poweredge_r440_system_board_inlet_temp_2 'iDRAC Power PowerEdge R440 System Board Inlet Temp' = unknown sensor.idrac_power_poweredge_r440_system_board_exhaust_temp_2 'iDRAC Power PowerEdge R440 System Board Exhaust Temp' = unknown sensor.temp02_air_temp 'Temp02 Air Temp' = 70.5 sensor.temp02_air_humidity 'Temp02 Air Humidity' = 40.9 switch.fish_tank 'Fish Tank' = on sensor.temp03_air_temp 'Temp03 Air Temp' = 71.6 sensor.temp03_wifi_signal 'Temp03 WiFi Signal' = -63 sensor.idrac_power_poweredge_r530_system_board_inlet_temp 'iDRAC Power PowerEdge R530 System Board Inlet Temp' = 75 sensor.idrac_power_poweredge_r530_cpu1_temp 'iDRAC Power PowerEdge R530 CPU1 Temp' = 117 sensor.idrac_power_poweredge_r330_system_board_inlet_temp 'iDRAC Power PowerEdge R330 System Board Inlet Temp' = unknown sensor.idrac_power_poweredge_r330_cpu1_temp 'iDRAC Power PowerEdge R330 CPU1 Temp' = unknown media_player.aio 'None' = unavailable binary_sensor.aio 'None' = unavailable light.aio_screen 'Screen' = unavailable sensor.idrac_power_cpu1_temp 'CPU1 Temp' = unavailable sensor.idrac_power_system_board_inlet_temp 'System Board Inlet Temp' = unavailable sensor.idrac_power_system_board_exhaust_temp 'System Board Exhaust Temp' = unavailable sensor.idrac_power_system_board_inlet_temp_2 'System Board Inlet Temp' = unavailable sensor.idrac_power_cpu1_temp_2 'CPU1 Temp' = unavailable sensor.idrac_power_system_board_inlet_temp_3 'System Board Inlet Temp' = unavailable sensor.idrac_power_cpu1_temp_3 'CPU1 Temp' = unavailable input_select.living_room_state 'Living Room Lights' = all-off scene.living_room_good_night 'Goodnight' = 2024-03-04T05:06:42.817971+00:00 sensor.temp02_air_temp 'Megans Room Temperature' = 70.5 sensor.temp03_air_temp 'Andrews Room Temperature' = 71.6 media_player.aio 'AIO' = unavailable media_player.aio 'All In One' = unavailable media_player.aio 'Living Room' = unavailable USER:What is the temperature in megans room

### Response: 8:32PM DBG Model already loaded in memory: Home-3B-v3.q5_k_m.gguf 8:32PM DBG Model 'Home-3B-v3.q5_k_m.gguf' already loaded 8:32PM DBG GRPC(Home-3B-v3.q5_k_m.gguf-127.0.0.1:34197): stdout {"timestamp":1709584351,"level":"INFO","function":"launch_slot_with_data","line":884,"message":"slot is processing task","slot_id":0,"task_id":21} 8:32PM DBG GRPC(Home-3B-v3.q5_k_m.gguf-127.0.0.1:34197): stdout {"timestamp":1709584351,"level":"INFO","function":"update_slots","line":1783,"message":"kv cache rm [p0, end)","slot_id":0,"task_id":21,"p0":0} 8:32PM DBG GRPC(Home-3B-v3.q5_k_m.gguf-127.0.0.1:34197): stdout {"timestamp":1709584362,"level":"INFO","function":"print_timings","line":327,"message":"prompt eval time = 8722.52 ms / 1466 tokens ( 5.95 ms per token, 168.07 tokens per second)","slot_id":0,"task_id":21,"t_prompt_processing":8722.52,"num_prompt_tokens_processed":1466,"t_token":5.94987721691678,"n_tokens_second":168.07069516607584} 8:32PM DBG GRPC(Home-3B-v3.q5_k_m.gguf-127.0.0.1:34197): stdout {"timestamp":1709584362,"level":"INFO","function":"print_timings","line":341,"message":"generation eval time = 1544.09 ms / 15 runs ( 102.94 ms per token, 9.71 tokens per second)","slot_id":0,"task_id":21,"t_token_generation":1544.09,"n_decoded":15,"t_token":102.93933333333332,"n_tokens_second":9.71445964937277} 8:32PM DBG GRPC(Home-3B-v3.q5_k_m.gguf-127.0.0.1:34197): stdout {"timestamp":1709584362,"level":"INFO","function":"print_timings","line":351,"message":" total time = 10266.61 ms","slot_id":0,"task_id":21,"t_prompt_processing":8722.52,"t_token_generation":1544.09,"t_total":10266.61} 8:32PM DBG GRPC(Home-3B-v3.q5_k_m.gguf-127.0.0.1:34197): stdout {"timestamp":1709584362,"level":"INFO","function":"update_slots","line":1594,"message":"slot released","slot_id":0,"task_id":21,"n_ctx":2048,"n_past":1480,"n_system_tokens":0,"n_cache_tokens":1481,"truncated":false} 8:32PM DBG Response: {"created":1709584178,"object":"chat.completion","id":"b3c7be7c-abc6-459d-b877-d94c2ef806e1","model":"Home-3B-v3.q5_k_m.gguf","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"\nThe temperature in Megans Room is 70.5.\n```"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}} [10.27.200.5]:52542 200 - POST /v1/chat/completions

acon96 commented 8 months ago

Because you are using LocalAI + the chat completions endpoint, you need to set the prompt template to Zephyr in LocalAI, not Home Assistant. Please refer to the discussion tab if you need further help configuring the component.