issue with enphase gateways becoming unresponsive

[davmoz]

There is an issue with enphase gateways becoming unresponsive, which leads to the gateway closing the connection and opening a perpetual task to open the device again. investigate this..

[davmoz]

One issue seems to be that the Enphase IQ gateway just closes the connection for some reason.. From 0123a8dfd510406bee:

[12/2/2024, 5:53:09 PM] ERROR:root:Failed to execute task <server.tasks.harvest.Harvest object at 0x7fb35be3d0>: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

And we're not handling this case, so would be a good idea to do that, e.g. start a perpetual task.

[davmoz]

Another case (0123bcdf5a82b4f6ee) where the IQ gateway just stops responding?

[12/1/2024, 6:00:15 AM] DEBUG:server.tasks.harvest:Harvest from [00:1d:c0:b0:d9:67] took 4474 ms

and then ~5 minutes later

[12/1/2024, 6:06:15 AM] INFO:server.tasks.harvest:Device is closed make the final transport if there is anything in the barn
[12/1/2024, 6:06:15 AM] INFO:server.tasks.harvest:Creating transport for https://mainnet.srcful.dev/gw/data
[12/1/2024, 6:06:15 AM] WARNING:server.app.task_scheduler:Task (<server.tasks.harvestTransport.HarvestTransport object at 0x7f8d85fcd0>) is in the past by 299944 ms, adjusting time to now
[12/1/2024, 6:06:15 AM] WARNING:server.app.task_scheduler:Task (<server.tasks.openDevicePerpetualTask.DevicePerpetualTask object at 0x7f8d85f450>) is in the past by 270044 ms, adjusting time to now
[12/1/2024, 6:06:15 AM] INFO:server.tasks.openDevicePerpetualTask:*************************************************************
[12/1/2024, 6:06:15 AM] INFO:server.tasks.openDevicePerpetualTask:******************** DevicePerpetualTask ********************
[12/1/2024, 6:06:15 AM] INFO:server.tasks.openDevicePerpetualTask:*************************************************************
[12/1/2024, 6:06:16 AM] INFO:server.tasks.harvestTransport:Response: <Response [200]>

This also happened in between:

[12/1/2024, 6:43:21 AM] ERROR:server.app.blackboard:org.freedesktop.DBus.Error.UnknownMethod: Object does not exist at path “/org/freedesktop/NetworkManager/AccessPoint/5316”

The second issue here is why the DevicePerpetualTask did not open a new connection.

[davmoz]

Another case (01239dde34631479ee):

[12/2/2024, 7:31:40 PM] DEBUG:server.tasks.harvest:Harvest from [20:d7:78:8d:7b:e3] took 7024 ms

[12/2/2024, 7:32:40 PM] ERROR:root:Failed to execute task <server.tasks.harvest.Harvest object at 0x7f7e85bb50>: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))