Unexpected error fetching FusionSolarOpenAPIDeviceRealKpiType data: 'NoneType' object is not iterable

[MrDadoo]

Describe the bug When querying FusionSolarOpenAPIDeviceRealKpiType for type ID: 47, i get a "None" data response. The code tries to iterate upon it and gets a "'NoneType' object is not iterable" exception.

Debug information

2024-01-30 09:39:28.895 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] FusionSolarOpenAPIDeviceRealKpiType Fetching data for type ID: 47 2024-01-30 09:39:29.254 DEBUG (SyncWorker_11) [custom_components.fusion_solar.fusion_solar.openapi.openapi_api] JSON data for https://eu5.fusionsolar.huawei.com/thirdData/getDevRealKpi: {'data': None, 'failCode': 0, 'message': None, 'params': {'currentTime': 1706603969231, 'devIds': '1000000137278685', 'devTypeId': 47}, 'success': True} 2024-01-30 09:39:29.259 ERROR (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Unexpected error fetching FusionSolarOpenAPIDeviceRealKpiType data: 'NoneType' object is not iterable Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 300, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/fusion_solar/device_real_kpi_coordinator.py", line 65, in _async_update_data for response_data in response: TypeError: 'NoneType' object is not iterable 2024-01-30 09:39:29.268 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Finished fetching FusionSolarOpenAPIDeviceRealKpiType data in 0.373 seconds (success: False) 2024-01-30 09:39:28.895 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] FusionSolarOpenAPIDeviceRealKpiType Fetching data for type ID: 47 2024-01-30 09:39:29.254 DEBUG (SyncWorker_11) [custom_components.fusion_solar.fusion_solar.openapi.openapi_api] JSON data for https://eu5.fusionsolar.huawei.com/thirdData/getDevRealKpi: {'data': None, 'failCode': 0, 'message': None, 'params': {'currentTime': 1706603969231, 'devIds': '1000000137278685', 'devTypeId': 47}, 'success': True} 2024-01-30 09:39:29.259 ERROR (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Unexpected error fetching FusionSolarOpenAPIDeviceRealKpiType data: 'NoneType' object is not iterable Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 300, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/fusion_solar/device_real_kpi_coordinator.py", line 65, in _async_update_data for response_data in response: TypeError: 'NoneType' object is not iterable

Answer the questions below before submitting a new issue.

• Are you using the latest version of Home Assistant? Yes
• Are you using the latest version of this integration? Yes
• Have you looked into the closed issues and did not find a similar issue? Yes
• Are you using Kiosk or Northbound / OpenAPI mode?
• If you are using Kiosk mode. Have you checked that the url is still valid? See I am using the kiosk mode, but the data is not updating
• If you ar using Northbound / OpenAPI mode. Check if your credentials are still valid. See How can I verify my Northbound API / OpenAPI credentials?
• Have you checked the logs for any info?

To Reproduce Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

[tijsverkoyen]

@MrDadoo would you be able to apply the changes from https://github.com/tijsverkoyen/HomeAssistant-FusionSolar/pull/146 in your local Home Assistant installation and report back if this resolves your issue.

[MrDadoo]

Does not solve it. Debug printout below.

As we actually are getting a None value in the json_data[ATTR_DATA] i don't see why we are rising an exception on that condition.

"data":null,

Shouldn't we ust ignore the content if its null?

2024-01-31 18:19:37.521 ERROR (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Unexpected error fetching FusionSolarOpenAPIDeviceRealKpiType data: Retrieving the data failed. Raw response: {"data":null,"failCode":0,"message":null,"params":{"currentTime":1706721577492,"devIds":"1000000137278685","devTypeId":47},"success":true}
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 300, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/fusion_solar/device_real_kpi_coordinator.py", line 54, in _async_update_data
    response = await self.hass.async_add_executor_job(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/fusion_solar/fusion_solar/openapi/openapi_api.py", line 157, in get_dev_real_kpi
    response = self._do_call(url, json)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/fusion_solar/fusion_solar/openapi/openapi_api.py", line 199, in _do_call
    raise FusionSolarOpenApiError(f'Retrieving the data failed. Raw response: {response.text}')
custom_components.fusion_solar.fusion_solar.openapi.openapi_api.FusionSolarOpenApiError: Retrieving the data failed. Raw response: {"data":null,"failCode":0,"message":null,"params":{"currentTime":1706721577492,"devIds":"1000000137278685","devTypeId":47},"success":true}
2024-01-31 18:19:37.537 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Finished fetching FusionSolarOpenAPIDeviceRealKpiType data in 0.373 seconds (success: False)

[MrDadoo]

Might it be so that I don't have Realtime Data enabled on my openapi account?

[MrDadoo]

Fetching data for type ID: 1 seems to work. After that line in the log

> 2024-01-31 18:54:23.456 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] FusionSolarOpenAPIDeviceRealKpiType Fetching data for type ID: 1
> 2024-01-31 18:54:24.009 INFO (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Fetching FusionSolarOpenAPIDeviceRealKpiType data recovered
> 2024-01-31 18:54:24.010 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Finished fetching FusionSolarOpenAPIDeviceRealKpiType data in 0.554 seconds (success: True)

my sensor values are updated.

But after "Fetching data for type ID: 47" all sensor data gets Invalid status and log shows

> 2024-01-31 18:55:27.164 DEBUG (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] FusionSolarOpenAPIDeviceRealKpiType Fetching data for type ID: 47
> 2024-01-31 18:55:27.495 ERROR (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Unexpected error fetching FusionSolarOpenAPIDeviceRealKpiType data: Retrieving the data failed. Raw response: {"data":null,"failCode":0,"message":null,"params":{"currentTime":1706723727464,"devIds":"1000000137278685","devTypeId":47},"success":true}
>   File "/config/custom_components/fusion_solar/device_real_kpi_coordinator.py", line 54, in _async_update_data

[tobyknobi]

After Huawei has updated there server. I have no data. I have no time to look into it yet to see what is happening

[danielhunt]

After Huawei has updated there server. I have no data. I have no time to look into it yet to see what is happening

Noticing the exact same here - I'm missing data for the past 3 days (since their server updates on 30th Jan)

[danielhunt]

After Huawei has updated there server. I have no data. I have no time to look into it yet to see what is happening

Noticing the exact same here - I'm missing data for the past 3 days (since their server updates on 30th Jan)

Actually - it turns out that data has started flowing again shortly after I sent that message.

And I think the reason is that I logged into FusionSolar's web app and accepted the privacy notice change that popped up...

[tobyknobi]

After Huawei has updated there server. I have no data. I have no time to look into it yet to see what is happening

Noticing the exact same here - I'm missing data for the past 3 days (since their server updates on 30th Jan)

Actually - it turns out that data has started flowing again shortly after I sent that message.

And I think the reason is that I logged into FusionSolar's web app and accepted the privacy notice change that popped up...

Unfortunately I still don't have a flow off data. Just updated the integration and ha, but still nothing. I even logged out of the app.

[danielhunt]

And I think the reason is that I logged into FusionSolar's web app and accepted the privacy notice change that popped up...

Unfortunately I still don't have a flow off data. Just updated the integration and ha, but still nothing. I even logged out of the app.

And have you accepted the new privacy policy after logging into the FusionSolar web app?

[danielhunt]

Actually I'm not sure what's going on. I definitely have data but I also see this in the logs:

2024-02-02 21:14:45.645 ERROR (MainThread) [custom_components.fusion_solar.device_real_kpi_coordinator] Error requesting FusionSolarOpenAPIDeviceRealKpiType data: 504 Server Error: Gateway Timeout for url: https://eu5.fusionsolar.huawei.com/thirdData/getDevRealKpi

So there's clearly still a problem

[tobyknobi]

And I think the reason is that I logged into FusionSolar's web app and accepted the privacy notice change that popped up...

Unfortunately I still don't have a flow off data. Just updated the integration and ha, but still nothing. I even logged out of the app.

And have you accepted the new privacy policy after logging into the FusionSolar web app?

Was looking for it. That's why I logged out but didn't get it.

[MrDadoo]

I've been able to play with POSTMAN and the API

The getDevRealKpi for ID 47 (in my setup) is the problematic one and it returns the following.

{
    "data": null,
    "failCode": 0,
    "message": null,
    "params": {
        "currentTime": 1706915299417,
        "devIds": "100000013XXXXXX",
        "devTypeId": 47
    },
    "success": true
}

I guess that the HA integration shall accept that devices have no data (or eventually I don't have access) and not make the other devices invalid. Can that be handled?

[tobyknobi]

It looks like mine is working again. I did a recreate container in portainer (although it did this before) and now it looks like it's working again. (did not even pull a new image) Well I was a bit to soon. I only see data flowing from battery state and consumption. But no flow of solar production

[MrDadoo]

Tried to do some if statements before the for response_data in resonse to skip iterating if response was None Also tried a catchall - but it all ended up in that all my sensordata got invalid.

Reason behind my problems was that I did not have proper rights on my account. It is corrected now and I get some meter data.

        except FusionSolarOpenApiAccessFrequencyTooHighError as e:
            self.skip = True
            return False
        except FusionSolarOpenApiError as error:
            raise UpdateFailed(f'OpenAPI Error: {error}')

        for response_data in response:
            key = f'{DOMAIN}-{response_data[ATTR_DEVICE_REAL_KPI_DEV_ID]}'
            data[key] = response_data[ATTR_DEVICE_REAL_KPI_DATA_ITEM_MAP]

        return data

As for now, I can see my smart meters phase currents using postman, but in ha, the sensor data seems to be a total sum of all sensor currents; I have only one current in the Meter-1 device..

I've browsed the integration code and it looks like the individual measurements should be available, but I cannot see them. Might my Meter-1 devcice have been wrongly when it was set up as I received wrong data in the beginning.

[tijsverkoyen]

getDevRealKpi is used to update multiple entities. So that is why "all" entities are unavailable.

I have created https://github.com/tijsverkoyen/HomeAssistant-FusionSolar/pull/148 which seems to resolve this. As it will not update any entity if the data object is empty.