Automation error mqtt.publish unknown service

[pokornyIt]

The problem

The "unknown service: mqtt.publish" error started popping up. This error occasionally appears for a few seconds. The error is triggered by activity in the automation. This did not happen in the previous version (2022.12.0).

I use automation to set external temperature on Aqura Smart Radiator Thermostat E1. MQTT broker is a standard HA add-on: Mosquitto broker (6.1.3).

Error message in the log:

Logger: homeassistant.components.automation.tvr_test_external_thermostat
Source: helpers/script.py:409
Integration: Automation ([documentation](https://www.home-assistant.io/integrations/automation), [issues](https://github.com/home-assistant/home-assistant/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+automation%22))
First occurred: 21:02:22 (12 occurrences)
Last logged: 23:15:35

    TRV - Zdenek external thermostat: If at step 1: Error executing script. Service not found for call_service at pos 1: Unable to find service mqtt.publish
    TRV - Zdenek external thermostat: Error executing script. Service not found for if at pos 1: Unable to find service mqtt.publish

Problem code parts in automatization:

    - service: mqtt.publish
      data:
        topic: zigbee2mqtt/Aqara SRT-E1 04/set/sensor_temp
        payload_template: '{{ (states(''sensor.sonoff_temp_humidity_05_temperature'')|float(0)|round(1))
          }}'

More people report a similar issue with other device types => https://community.home-assistant.io/t/unknown-service-mqtt-publish-but-only-for-a-second/477242

What version of Home Assistant Core has the issue?

core-2022.12.7

What was the last working version of Home Assistant Core?

core-2022.12.0

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Automation

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

- id: '1671572053054'
  alias: TRV - Maty thermostat external
  trigger:
  - platform: state
    entity_id:
    - sensor.sonoff_temp_humidity_05_temperature
  condition:
  - condition: or
    conditions:
    - condition: template
      value_template: "{{ (state_attr('climate.aqara_srt_e1_04','current_temperature')
        -\n (states('sensor.sonoff_temp_humidity_05_temperature')|float(0)|round(1)))
        | abs > 0.1 }}"
    - condition: template
      value_template: '{{ states(''sensor.aqara_srt_e1_04'') != ''external'' }}'
  action:
  - if:
    - condition: or
      conditions:
      - condition: template
        value_template: "{{ (state_attr('climate.aqara_srt_e1_04','current_temperature')
          -\n (states('sensor.sonoff_temp_humidity_05_temperature')|float(0)|round(1)))
          | abs > 0.5 }}"
      - condition: template
        value_template: '{{ states(''sensor.aqara_srt_e1_04'') != ''external'' }}'
    then:
    - service: mqtt.publish
      data:
        topic: zigbee2mqtt/Aqara SRT-E1 04/set/sensor_temp
        payload_template: '{{ (states(''sensor.sonoff_temp_humidity_05_temperature'')|float(0)|round(1))
          }}'
    - device_id: a71867825e29bccf48bf2612486c2d23
      domain: select
      entity_id: select.aqara_srt_e1_04_sensor
      type: select_option
      option: external
  - service: mqtt.publish
    data:
      topic: zigbee2mqtt/Aqara SRT-E1 04/set/sensor_temp
      payload_template: '{{ (states(''sensor.sonoff_temp_humidity_05_temperature'')|float(0)|round(1))
        }}'
  mode: single

Anything in the logs that might be useful for us?

2022-12-20 23:15:35.383 ERROR (MainThread) [homeassistant.components.automation.trv_jira] TRV - Jira external thermostat: If at step 1: Error executing script. Service not found for call_service at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.384 ERROR (MainThread) [homeassistant.components.automation.trv_sima_external_thermostat] TRV - Sima external thermostat: If at step 1: Error executing script. Service not found for call_service at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.384 ERROR (MainThread) [homeassistant.components.automation.trv_maty_thermostat_external] TRV - Maty thermostat external: If at step 1: Error executing script. Service not found for call_service at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.385 ERROR (MainThread) [homeassistant.components.automation.tvr_test_external_thermostat] TRV - Zdenek external thermostat: If at step 1: Error executing script. Service not found for call_service at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.397 ERROR (MainThread) [homeassistant.components.automation.trv_jira] TRV - Jira external thermostat: Error executing script. Service not found for if at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.398 ERROR (MainThread) [homeassistant.components.automation.trv_sima_external_thermostat] TRV - Sima external thermostat: Error executing script. Service not found for if at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.400 ERROR (MainThread) [homeassistant.components.automation.trv_maty_thermostat_external] TRV - Maty thermostat external: Error executing script. Service not found for if at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:35.406 ERROR (MainThread) [homeassistant.components.automation.tvr_test_external_thermostat] TRV - Zdenek external thermostat: Error executing script. Service not found for if at pos 1: Unable to find service mqtt.publish
2022-12-20 23:15:37.701 ERROR (SyncWorker_4) [homeassistant] Error doing job: Task was destroyed but it is pending!

Additional information

No response

[home-assistant[bot]]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (automation) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `automation` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign automation` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

automation documentation automation source _{^{(message by IssueLinks)}}

[home-assistant[bot]]

Hey there @emontnemery, @jbouwh, mind taking a look at this issue as it has been labeled with an integration (mqtt) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `mqtt` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign mqtt` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

mqtt documentation mqtt source _{^{(message by IssueLinks)}}

[jbouwh]

When the mqtt.publish service is not available, is this during start-up of HA? This error can occur when the MQTT config entry has not been loaded and automations try to call the service.

[pokornyIt]

Hi, I see this when the MQTT runs more than 5 hours. This issue is not dependent on the system start.

[jbouwh]

Could you check your logging to see if anything interesting there?

[pokornyIt]

Nothing more than I added to the issue.

[RoyTrenneman]

Hi, On my side, MQTT doesn't start at all. I can see in log Setup failed for mqtt: Unable to import component: cannot import name 'MQTT_DATA_DEVICE_TRACKER_LEGACY' from 'homeassistant.components.mqtt.const' (/srv/homeassistant/lib/python3.10/site-packages/homeassistant/components/mqtt/const.py) mqttpublisher: Error executing script. Service not found for call_service at pos 1: Unable to find service mqtt.publish But I dont have any device tracker configured through MQTT. So, I re added MQTT_DATA_DEVICE_TRACKER_LEGACY in const.py in order to workaround.

[jbouwh]

Can you find out what tries to import it? Are you running the latest version of Home Assistant?

[RoyTrenneman]

Yes I am using the last release. I tried to find in my conf.yaml and automation.yaml but I did not find. Is there a way to find what try to import this legacy tracker? Surely an old test when I used MQTT tracker, but not now, for a long time.

[jbouwh]

Is there a way to find what try to import this legacy tracker?

I searched the code base but could not find it. Are you using any custom integrations?

[RoyTrenneman]

I searched the code base but could not find it. Are you using any custom integrations?

I removed all custom integrations I had. Unfortunately, same error when starting HA homeassistant.setup] Setup failed for mqtt: Unable to import component: cannot import name 'MQTT_DATA_DEVICE_TRACKER_LEGACY' from 'homeassistant.components.mqtt.const' (/srv/homeassistant/lib/python3.10/site-packages/homeassistant/components/mqtt/const.py)

[RoyTrenneman]

I use my mobile as a device_tracker through the HA apps from google store (v2022.12.0-full). Is there a link ? I think it's not a MQTT tracker, isn't it?

[jbouwh]

The mobile app should not have influence on ha core I would think. Somehow this import is still requested. But it is not in dev.

[RoyTrenneman]

The mobile app should not have influence on ha core I would think. Somehow this import is still requested. But it is not in dev.

I agree, where the workaround seems work!

[jbouwh]

The const was removed with https://github.com/home-assistant/core/pull/82102 and was released with 2022.12. Interesting is that you indicated that it did not happen with this version. It seems the discussion on HA community was going some time ago. Since it is MQTT that does not set up, may be something of old code was left behind.

[RoyTrenneman]

. Interesting is that you indicated that it did not happen with this version

Yes, I confirm that the issue starts at 2022.12.

[jbouwh]

I have two questions:

1. Can you check you have if this file is still present in your install? homeassistant/components/mqtt/device_tracker/schema_yaml.py. B.T.W. This file should have been removed with the upgrade to HA Core 2022.12, you can safely remove it.

2. What method do you use to upgrade Home Assistant core?

[RoyTrenneman]

Can you check you have if this file is still present in your install?

Yes it is present

What method do you use to upgrade Home Assistant core?

I use a virtual env with python 3.10 in /srv/homeassistant then I launch: python3.10 -m pip install /home/homeassistant/core/

[jbouwh]

I saw https://github.com/pypa/pip/issues/5020 but I do not know it applies to your case (it is about --ignore-installed). Perhaps in some way something went wrong when updating HA core, leaving an old source file behind.

[RoyTrenneman]

interesting! I continue my investigations, I will keep you informed

[RoyTrenneman]

I removed pip cache and all site-packages from /srv/homeassistant, but no change, the file homeassistant/components/mqtt/device_tracker/schema_yaml.py is still installed.

[jbouwh]

You mean that pip did not remove all files, or that when installing a clean version schema_yaml.py is installed too? When I fetch a fresh docker container for home-assistant/homeassistant the file is not in there. Assuming you are using pip, what options are you using when upgrading? Dit you try to remove Home Assistant and check then if there are files left behind?

[RoyTrenneman]

I mean that when installing a clean version schema_yaml.py is installed too. Maybe something in my configurations.yaml triggers the device_tracker legacy installation?

Assuming you are using pip, what options are you using when upgrading?

python3.10 -m pip install /home/homeassistant/core/ from a fresh venv.

Dit you try to remove Home Assistant and check then if there are files left behind?

Yes I did; no effect

[jbouwh]

What are exactly the steps so I can reproduce this. I am not very familiar with venv's.

[jbouwh]

I have tried to reproduce installing homeassistant using a fresh venv. The result is that I could not find the old files. So the pip sources seem fine to me. It must have been caused during an upgrade that was not removing the old files. May once the --ignore-installed option was used, see also https://github.com/pypa/pip/issues/5020 ?

[jbouwh]

If have done a final test in a venv and installed 2022.11.5 (pip3 install homeassistant==2022.11.5 --upgrade) that still contains the file ./device_tracker/schema_yaml.py. Then I executed in the venv pip3 install homeassistant --upgrade. The old file and the folder /device_tracker was removed correctly. I think this is not an issue with our packages.

[RoyTrenneman]

ok, thanks for your tests! I am puzzled, I did exactly the same as you, and the file./device_tracker/schema_yaml.py gone back. I think the key device_tracker in my config.yaml triggers this old install, what do you think about that?

[jbouwh]

ok, thanks for your tests! I am puzzled, I did exactly the same as you, and the file./device_tracker/schema_yaml.py gone back. I think the key device_tracker in my config.yaml triggers this old install, what do you think about that?

It does not seem to me that your config is triggering source files to change ;-). MQTT is a core part of homeassistant. I cannot reproduce your issue.

[jbouwh]

It seems this is not an issue with Home Assistant or the MQTT integration but with an install that had issues. Closing this issue here.

[davidjb]

I had the same issue upon upgrading to 2022.12.9 from 2022.11.5 (which has been upgraded from every previous monthly release) using a venv, Git-based source install (e.g. git clone and pip install .). My installs correctly installed device_tracker.py but retained the device_tracker/ directory and its contents (__init__.py, __pycache__/, schema_discovery.py, schema_yaml.py) as well, thus causing problems because of overlapping namespaces and the latter taking precedence over the former.

The device_tracker/ directory was most definitely gone in my local cloned repo, but kept returning upon pip install, despite either running pip uninstall homeassistant or rm -rf'ing the egg directory.

The issue stemmed from the build/ directory in my git repo directory (where I ran pip install) containing remnants of old installations due to pip not cleaning it up, despite indications that it should be by default. I wasn't using the --no-clean option and am running the latest pip so I'm guessing there's a bug there.

In any case, rm -rf build and reinstalling with pip install removed the old files for good and fixed the issue.