Environnement Canada : Unable to retrieve Environment Canada AQHI

[Guyohms]

The problem

This errors shows on each restart of HA.

Source: components/environment_canada/__init__.py:66
Integration: Environment Canada (documentation, issues)
First occurred: 15:53:23 (1 occurrences)
Last logged: 15:53:23
Unable to retrieve Environment Canada AQHI

It also causes a slowdown of the starting process.

And the related sensor sensor.town_aqhi is Unavailable

I have this issue since a few weeks at least.

What version of Home Assistant Core has the issue?

2022.3.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Environnement Canada

Link to integration documentation on our website

https://www.home-assistant.io/integrations/environment_canada/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant[bot]]

environment_canada documentation environment_canada source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @gwww, @michaeldavie, mind taking a look at this issue as it has been labeled with an integration (environment_canada) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[gwww]

We will need logs to understand further. Can you add the following to your configuration.yaml and share the logs.

Additionally, has this ever worked for you? Is it just the AQHI sensor or others?

logger:
  default: warning
  logs:
     env_canada: debug
     homeassistant.components.environment_canada: debug

[Guyohms]

Everything else works as intended.

Here's the log:

2022-03-13 18:29:03 INFO (MainThread) [homeassistant.components.environment_canada] Fetching environment_canada weather data recovered
2022-03-13 18:29:03 DEBUG (MainThread) [homeassistant.components.environment_canada] Finished fetching environment_canada weather data in 2.714 seconds (success: True)
2022-03-13 18:29:13 INFO (MainThread) [homeassistant.components.environment_canada] Fetching environment_canada radar data recovered
2022-03-13 18:29:13 DEBUG (MainThread) [homeassistant.components.environment_canada] Finished fetching environment_canada radar data in 9.871 seconds (success: True)
2022-03-13 18:29:13 DEBUG (MainThread) [homeassistant.components.environment_canada] Finished fetching environment_canada AQHI data in 0.529 seconds (success: False)
2022-03-13 18:29:13 WARNING (MainThread) [homeassistant.components.environment_canada] Unable to retrieve Environment Canada AQHI

[gwww]

Thanks for the logs. Unfortunately, I did not learn anything from them. Onto questions...

• What station ID are you using (or coordinates) for the Environment Canada integration?
• Does this happen all the time or just occasionally?
• When did this start?

[Guyohms]

Yeah the debug log isn't very helping.

I am using station ID QC/s0000202 It happens all the time. Even if I disable the AQHI sensor. In fact I only have the weather and the camera entities enabled at the moment. I'm not sure when this started. Maybe since the 2022.02 update

[gwww]

A couple of things. I can reproduce! Also I've added some logs to the base library, which will help in the future.

I don't know why yet, but reproducing is key!

Here are the logs that I'm seeing.

2022-03-15 11:06:45 DEBUG (MainThread) [env_canada.ec_weather] update(): station QC/s0000202 lat 45.320000 lon -73.270000
2022-03-15 11:06:51 DEBUG (MainThread) [env_canada.ec_aqhi] get_aqhi_regions() started
2022-03-15 11:06:51 DEBUG (MainThread) [env_canada.ec_aqhi] get_aqhi_regions(): found 134 regions
2022-03-15 11:06:51 DEBUG (MainThread) [env_canada.ec_aqhi] update() closest region returned: zone_id 'que' region_id 'EHHUN'
2022-03-15 11:06:53 DEBUG (MainThread) [homeassistant.components.environment_canada] Finished fetching environment_canada AQHI data in 1.798 seconds (success: False)
2022-03-15 11:06:53 WARNING (MainThread) [homeassistant.components.environment_canada] Unable to retrieve Environment Canada AQHI

The particular error, found through more logging is:

raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url=URL('https://dd.weather.gc.ca/air_quality/aqhi/que/observation/realtime/xml/AQ_OBS_EHHUN_CURRENT.xml')

[gwww]

What is happening is that there is no air quality information for the Quebec region. I don't know why, but in the end it doesn't matter. The only question is how to be robust when there are errors.

So first, this error does not slow down startup. In fact if anything it might speed it up, if only every so slightly. What is happening is that when retrieving AQHI info a 404 (HTTP error) is being returned. This happens very quickly. The slow startup is retrieving radar information. You can see the timings in the logs you provided above.

The problem occurs because on startup weather, radar, and air quality are all retrieved regardless of whether there are any entities enabled. This is, as far as I know, the only "right" way to startup an integration. There is a possible fix there, but will require help from the HA core devs.

The simplest and most likely the best solution is to change the warning to an informational message so that the message does not raise concerns; i.e.: it is expected that for some regions that there is no AQHI data so here is an informational message telling you that.

What do you think of that solution? @michaeldavie - your thoughts?

[Guyohms]

Great explanation! Thank you I think that hiding the warning is fine but then the AQHI sensor will still be shown with a warning () in the entities list. That can be misleading or prone to questions.

Maybe automatically disable the sensor if AQHI returns an error and specify that in the same INFO log?

[gwww]

I don't believe that I can disable an entity on error. What I can do is when the integration is configured, create the AQHI entity as disabled (we already do that for radar). That way people will have to enable it manually... and deal with the error. I also think that the documentation should be updated to explain a few of these subtleties.

[Guyohms]

Another fix would be to not ask for AQHI for Québec, so no error and no "bad" entity created. But I don't know if it's best practice to manage exceptions like that.

If not, your idea seems to deal with the error correctly by hiding the warning and disabling the sensor.

I agree that a note should be added to the docs explaining this.

[michaeldavie]

It looks like there are no air quality observations for Quebec but there are forecasts: https://weather.gc.ca/airquality/pages/provincial_summary/qc_e.html. If we err on the side of returning something, we could fallback to the forecast value. This is what the temperature sensor currently does if there is no observation data.

[gwww]

I just discovered this:

Air quality observations are provided by provinces and municipalities. Provincial jurisdictions also control how observations are communicated to the public. Quebec did not agree to the publication of current air quality in the form of an air quality health index (AQHI). This explains why no observation are available at the address: https://dd.weather.gc.ca/air_quality/aqhi/que/observation/. However, the Ministère du Développement durable, Environnement et Lutte contre les changements climatiques (MDDELCC) and Ville de Montreal also redistribute some of their data on [the American AirNow portal](https://www.epa.gov/outdoor-air-quality-data/download-daily-data).

From https://eccc-msc.github.io/open-data/msc-data/aqhi/readme_aqhi-datamartxml_en/. So no go on AQHI for Quebec!

I'm going ahead with the simple fix for now. Changing the warning to info and when adding the integration AQHI is created as disabled by default. Another PR will be needed to update docs.