Fronius integration fails to initialize after latest home assistant update

trdischat commented 2 years ago

The problem

After updating home assistant from 2021.11.5 to 2021.12.7, the Fronius integration fails during setup.

What version of Home Assistant Core has the issue?

core-2021.12.7

What was the last working version of Home Assistant Core?

core-2021.11.5

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Fronius

Link to integration documentation on our website

https://www.home-assistant.io/integrations/fronius

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Error setting up entry SolarNet Datalogger at 192.168.21.5 for fronius
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 313, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/usr/src/homeassistant/homeassistant/components/fronius/__init__.py", line 41, in async_setup_entry
    await solar_net.init_devices()
  File "/usr/src/homeassistant/homeassistant/components/fronius/__init__.py", line 103, in init_devices
    self.system_device_info = await self._create_solar_net_device()
  File "/usr/src/homeassistant/homeassistant/components/fronius/__init__.py", line 163, in _create_solar_net_device
    solar_net_device[ATTR_MODEL] = _logger_info["product_type"]["value"]
KeyError: 'product_type'

Additional information

I have commented out the configuration of this integration from my configuration.yaml, then deleted and re-added the integration through the GUI, but the error is unchanged.

probot-home-assistant[bot] commented 2 years ago

fronius documentation fronius source _{^{(message by IssueLinks)}}

probot-home-assistant[bot] commented 2 years ago

Hey there @nielstron, @farmio, mind taking a look at this issue as it has been labeled with an integration (fronius) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

farmio commented 2 years ago

Hi 👋! Thank you for reporting this issue!

What kind of Fronius device do you use? And what is it's current Software version?

Could you get me the output of curl http://<FRONIUS_IP>/solar_api/v1/GetLoggerInfo.cgi?

trdischat commented 2 years ago

I have a Fronius Datalogger Web recording data from a Fronius IG 3000 inverter. It runs on version 0 of the Fronius API. So to get the logger info, I have to run http://<FRONIUS_IP>/solar_api/GetLoggerInfo.cgi? instead, which yields this:

{
    "Head" : {
        "RequestArguments" : {},
        "Status" : {
            "Code" : 0,
            "Reason" : "",
            "UserMessage" : ""
        },
        "Timestamp" : "2021-12-31T08:06:51-08:00"
    },
    "Body" : {
        "LoggerInfo" : {
            "UniqueID" : "240.12345",
            "HWVersion" : "1.4A",
            "SWVersion" : "2.0.5-4",
            "TimezoneLocation" : "Tijuana",
            "TimezoneName" : "PST",
            "UTCOffset" : -28800,
            "DefaultLanguage" : "en",
            "CashFactor" : 0.42,
            "CashCurrency" : "&#85;&#83;&#68;",
            "CO2Factor" : 0.163,
            "CO2Unit" : "&#107;&#103;"
        }
    }
}

trdischat commented 2 years ago

"ProductID" is not defined in v0 of the API.

As a test, I revised line 163 to read like this:

solar_net_device[ATTR_MODEL] = "fronius-datalogger-web"

... and it now runs for my v0 logger. From the Fronius documentation, it appears that the Datalogger Web was the only v0 data logger device. Obviously, the line has to be written to work for both versions of the API.

farmio commented 2 years ago

Great, thanks for pointing that out! I'd say we use a generic "Datalogger" as default model or just make it optional.

@trdischat do you want to make the fixing PR yourself? It would be great to also have unit tests for a APIv0 system, but you seem to be the first one having such so I didn't have any data to create one before.

Happy new year 🥳

trdischat commented 2 years ago

It would be great to also have unit tests for a APIv0 system

Do you need more v0 material than what is already here?

https://github.com/nielstron/pyfronius/tree/master/pyfronius/tests

farmio commented 2 years ago

In HA we have fixtures for tests that are pyfronius transparent. So when there are changes in pyfronius we will see if they break the integration when adding the requirement update. See the fixtures directory in https://github.com/home-assistant/core/tree/dev/tests/components/fronius for the needed files (gen24 for minimal required).

trdischat commented 2 years ago

What data do you need for the fixtures? Attached as JSON files are all of the outputs that I could obtain from my v0 system.

fronius_outputs.zip

farmio commented 2 years ago

Thanks! I'll have a look. Some endpoints seem to be not supported by v0 (eg. power_flow and even api_version). Let's see if I can add a test for this.

trdischat commented 2 years ago

GetAPIVersion is not supported by v0 of the API. But as suggested by the v0 documentation, you can just assume v0 of the API if this command returns HTTP error 404. The fetch_api_version method in pyfronius already does this.

farmio commented 2 years ago

Yes, right. I'd be interested nevertheless how the Datalogger responds to not implement requests (not only getAPIversion). Does it just time out, 404, any other http status code or json-respond somehow? As I said, these tests aim to be pyfronius transparent.

This is what I used to generate the test data:

FRONIUS_IP=10.1.2.5
curl "http://$FRONIUS_IP/solar_api/GetAPIVersion.cgi" -o GetAPIVersion.json
curl "http://$FRONIUS_IP/solar_api/v1/GetMeterRealtimeData.cgi?Scope=System" -o GetMeterRealtimeData.json
curl "http://$FRONIUS_IP/solar_api/v1/GetStorageRealtimeData.cgi?Scope=System" -o GetStorageRealtimeData.json
curl "http://$FRONIUS_IP/solar_api/v1/GetOhmPilotRealtimeData.cgi?Scope=System" -o GetOhmPilotRealtimeData.json
curl "http://$FRONIUS_IP/solar_api/v1/GetPowerFlowRealtimeData.fcgi" -o GetPowerFlowRealtimeData.json
curl "http://$FRONIUS_IP/solar_api/v1/GetInverterInfo.cgi" -o GetInverterInfo.json
curl "http://$FRONIUS_IP/solar_api/v1/GetInverterRealtimeData.cgi?Scope=System" -o GetInverterRealtimeData_System.json
curl "http://$FRONIUS_IP/solar_api/v1/GetInverterRealtimeData.cgi?Scope=Device&DeviceId=1&DataCollection=CommonInverterData" -o GetInverterRealtimeData_Device_1.json
curl "http://$FRONIUS_IP/solar_api/v1/GetSensorRealtimeData.cgi?Scope=System" -o GetSensorRealtimeData.json
curl "http://$FRONIUS_IP/solar_api/v1/GetLoggerInfo.cgi" -o GetLoggerInfo.json
curl "http://$FRONIUS_IP/solar_api/v1/GetActiveDeviceInfo.cgi?DeviceClass=System" -o GetActiveDeviceInfo.json

you would probably have to remove the /v1 part of the urls.

trdischat commented 2 years ago

I removed the v1 from the URLs, ran the script, and this is what I got:

fronius_test.zip

In v0 of the API:

GetActiveDeviceInfo does not work with "System" as a DeviceClass
GetInverterRealtimeData uses DeviceIndex, not DeviceID, as a parameter
Scope is not a valid parameter for GetSensorRealtimeData
GetAPIVersion, GetMeterRealtimeData, GetStorageRealtimeData, GetOhmPilotRealtimeData, are GetPowerFlowRealtimeData are undefined and return HTTP error 404

home-assistant / core