krbaker / hass-sunpower

Home Assistant SunPower Integration using the local installer ethernet interface.
Apache License 2.0
63 stars 23 forks source link

Issue with one or both my PVS Units #81

Open hakk2020 opened 5 months ago

hakk2020 commented 5 months ago

Hi, I started experiencing issues getting data into Home Assistant from either one or both of my PVS units. Running this in the Browser shows all the details http:///cgi-bin/dl_cgi?Command=DeviceList every time and there are no networking issues I can see. Below is the output of the DevileList command from the PVS that's currently nor reporting in HA. I hope you can help me figure out the issue.

{ "devices": [ { "DETAIL": "detail", "STATE": "working", "STATEDESCR": "Working", "SERIAL": "ZT230585000549F2221", "MODEL": "PV Supervisor PVS6", "HWVER": "6.03", "SWVER": "2023.3, Build 61410", "DEVICE_TYPE": "PVS", "DATATIME": "2024,06,04,22,55,00", "dl_err_count": "0", "dl_comm_err": "141", "dl_skipped_scans": "0", "dl_scan_time": "1", "dl_untransmitted": "660904", "dl_uptime": "88591", "dl_cpu_load": "0.50", "dl_mem_used": "78608", "dl_flash_avail": "63019", "panid": 1411269642, "CURTIME": "2024,06,04,22,57,14" }, { "ISDETAIL": true, "SERIAL": "PVS6M23052221p", "TYPE": "PVS5-METER-P", "STATE": "working", "STATEDESCR": "Working", "MODEL": "PVS6M0400p", "DESCR": "Power Meter PVS6M23052221p", "DEVICE_TYPE": "Power Meter", "interface": "mime", "production_subtype_enum": "GROSS_PRODUCTION_SITE", "subtype": "GROSS_PRODUCTION_SITE", "SWVER": "3000", "PORT": "", "DATATIME": "2024,06,04,22,57,13", "ct_scl_fctr": "50", "net_ltea_3phsum_kwh": "2841.35", "p_3phsum_kw": "0.3833", "q_3phsum_kvar": "0.1575", "s_3phsum_kva": "0.46", "tot_pf_rto": "0.8363", "freq_hz": "60", "i_a": "1.9146", "v12_v": "240.2844", "CAL0": "50", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,14" }, { "ISDETAIL": true, "SERIAL": "PVS6M23052221c", "TYPE": "PVS5-METER-C", "STATE": "working", "STATEDESCR": "Working", "MODEL": "PVS6M0400c", "DESCR": "Power Meter PVS6M23052221c", "DEVICE_TYPE": "Power Meter", "interface": "mime", "subtype": "NOT_USED", "SWVER": "3000", "PORT": "", "DATATIME": "2024,06,04,22,57,13", "ct_scl_fctr": "100", "net_ltea_3phsum_kwh": "0", "p_3phsum_kw": "0", "q_3phsum_kvar": "0", "s_3phsum_kva": "0", "tot_pf_rto": "0", "freq_hz": "60", "i1_a": "0", "i2_a": "0", "v1n_v": "120.3094", "v2n_v": "119.9756", "v12_v": "240.2844", "p1_kw": "0", "p2_kw": "0", "neg_ltea_3phsum_kwh": "0", "pos_ltea_3phsum_kwh": "0", "CAL0": "100", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,14" }, { "ISDETAIL": true, "SERIAL": "E00122247100156", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247100156", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,56,58", "ltea_3phsum_kwh": "351.2553", "p_3phsum_kw": "0.0463", "vln_3phavg_v": "242.37", "i_3phsum_a": "0.19", "p_mppt1_kw": "0.049", "v_mppt1_v": "35.78", "i_mppt1_a": "1.37", "t_htsnk_degc": "30", "freq_hz": "60.01", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,14" }, { "ISDETAIL": true, "SERIAL": "E00122247100777", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247100777", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,56,58", "ltea_3phsum_kwh": "360.2006", "p_3phsum_kw": "0.0469", "vln_3phavg_v": "241.38", "i_3phsum_a": "0.19", "p_mppt1_kw": "0.0497", "v_mppt1_v": "35.77", "i_mppt1_a": "1.39", "t_htsnk_degc": "30", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,15" }, { "ISDETAIL": true, "SERIAL": "E00122247101126", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247101126", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,56,58", "ltea_3phsum_kwh": "353.2716", "p_3phsum_kw": "0.0458", "vln_3phavg_v": "241.45", "i_3phsum_a": "0.19", "p_mppt1_kw": "0.0492", "v_mppt1_v": "35.79", "i_mppt1_a": "1.37", "t_htsnk_degc": "31", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,15" }, { "ISDETAIL": true, "SERIAL": "E00122247106541", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247106541", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,56,58", "ltea_3phsum_kwh": "346.2702", "p_3phsum_kw": "0.0457", "vln_3phavg_v": "241.72", "i_3phsum_a": "0.18", "p_mppt1_kw": "0.0493", "v_mppt1_v": "35.6", "i_mppt1_a": "1.38", "t_htsnk_degc": "30", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,15" }, { "ISDETAIL": true, "SERIAL": "E00122247106596", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247106596", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,56,58", "ltea_3phsum_kwh": "355.6719", "p_3phsum_kw": "0.0462", "vln_3phavg_v": "241.32", "i_3phsum_a": "0.19", "p_mppt1_kw": "0.0498", "v_mppt1_v": "35.7", "i_mppt1_a": "1.39", "t_htsnk_degc": "31", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,15" }, { "ISDETAIL": true, "SERIAL": "E00122247113775", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247113775", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,57,13", "ltea_3phsum_kwh": "344.5262", "p_3phsum_kw": "0.0455", "vln_3phavg_v": "242.48", "i_3phsum_a": "0.18", "p_mppt1_kw": "0.049", "v_mppt1_v": "35.8", "i_mppt1_a": "1.36", "t_htsnk_degc": "30", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,15" }, { "ISDETAIL": true, "SERIAL": "E00122247114265", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247114265", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,57,13", "ltea_3phsum_kwh": "344.6895", "p_3phsum_kw": "0.0454", "vln_3phavg_v": "242.05", "i_3phsum_a": "0.18", "p_mppt1_kw": "0.0498", "v_mppt1_v": "35.86", "i_mppt1_a": "1.39", "t_htsnk_degc": "30", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,15" }, { "ISDETAIL": true, "SERIAL": "E00122247114391", "TYPE": "SOLARBRIDGE", "STATE": "working", "STATEDESCR": "Working", "MODEL": "AC_Module_Type_H", "DESCR": "Inverter E00122247114391", "DEVICE_TYPE": "Inverter", "hw_version": "4407", "interface": "mime", "module_serial": "", "PANEL": "WAAREE-AKRA-400W", "slave": 0, "SWVER": "4.21.4", "PORT": "", "MOD_SN": "", "NMPLT_SKU": "", "DATATIME": "2024,06,04,22,57,13", "ltea_3phsum_kwh": "350.7921", "p_3phsum_kw": "0.046", "vln_3phavg_v": "241.91", "i_3phsum_a": "0.19", "p_mppt1_kw": "0.0507", "v_mppt1_v": "35.84", "i_mppt1_a": "1.41", "t_htsnk_degc": "31", "freq_hz": "60", "stat_ind": "0", "origin": "data_logger", "OPERATION": "noop", "CURTIME": "2024,06,04,22,57,16" } ], "result": "succeed" }

krbaker commented 5 months ago

@hakk2020 can you also check your logs? ( http:///config/logs or https:// ) and search for "sunpower". If there isn't much in there you can get even more detailed logs in the integration page (you'll probably want to filter those out if you capture them! image

In the mean time I will look at these in more detail.

hakk2020 commented 5 months ago

Thank you for responding so quickly! Coincidentally, Sunpower sent a tech out to troubleshoot one of my systems due to not reporting production. He said he replaced one of the CTs and upgraded the firmware, after which both systems started reporting to Home Assistant. However, a day later one of them stopped again. Looking at the Sunpower App, both systems are online and reporting. When I try to get to the one that's not reporting to Home Assistant, I get '403 Forbidden'. I wonder if this is an issue that's related to my PVS and not the integration. I'm attaching the integration log, in case you'd like to review. home-assistant_sunpower_2024-06-09T00-21-59.710Z.log

hakk2020 commented 5 months ago

Just an update, I rebooted the PVS twice. After the second reboot, it started reporting properly.

jtooley307 commented 5 months ago

When I run into this problem I reload the integration in HA.

hakk2020 commented 5 months ago

When I run into this problem I reload the integration in HA.

Thank you for the suggestion! Unfortunately, I tried reloading the integration, removing/adding the two systems. Nothing seems to stick. I get both systems to work for a few hours and then one of them stops reporting. It's not a networking issue, because I can access both systems via IP. If I keep only one system in the integration, it seems to work consistently. As far as I can tell, my problem exists only when the two systems are present in the integration.

krbaker commented 5 months ago

You mentioned "When I try to get to the one that's not reporting to Home Assistant, I get '403 Forbidden'. I wonder if this is an issue that's related to my PVS and not the integration"

I'm concerned that sunpower might have disabled this endpoint after some period of uptime (it would still allow installers to use it - which is its intended usecase). I'd love to get more info to be sure:

Can you 1) provide more information about the 403 error? (full HTTP transaction) where did you see the 403 error? 2) try a reboot and see how long it takes to start failing? Does rebooting always bring it back to working? 3) get an output of the DeviceList endpoint in both cases 403 and not 403 (the curl output with -vvv is probably good, I'd love to see curl fail as well as it would eliminate the integration from the possible issues) 4) explain how are you connecting to the PVS? are you using a proxy, ip routing or something else (is it possible the 403 is coming from a system in the middle)

hakk2020 commented 5 months ago

Hi, sorry for the delayed response. I appreciate your help. The main thing at the moment is that the 403 error was resolved after rebooting the PVS a couple of times. At present, I can access both my PVS systems using the DeviceList URL. Please see below for answers to your questions.

  1. Provide more information about the 403 error? (full HTTP transaction) where did you see the 403 error? The 403 error is no longer present
  2. Try a reboot and see how long it takes to start failing? Does rebooting always bring it back to working? As best as I can tell, after I reboot the system that's not accessible by the integration, sometimes the integration can access it for a few hours, sometimes not at all.
  3. Get an output of the DeviceList endpoint in both cases 403 and not 403 (the curl output with -vvv is probably good, I'd love to see curl fail as well as it would eliminate the integration from the possible issues) I'm providing the output from both systems in the form of text attachments. However, the 403 error is gone. Both systems are currently accessible via the DeviceList URL. See attached files.
  4. explain how are you connecting to the PVS? are you using a proxy, ip routing or something else (is it possible the 403 is coming from a system in the middle) I have two mini routers model GL-MT300N-V2. See following link for details. The routers have unique IPs on my network. On the other side, the routers are connected to the management port of the PVS. I do port forwarding from my network on port 8080 to port 80 on the PVS on the management ethernet interface. For example, IP 192.168.10.100:8080 on my network is port forwarded by the router to 172.27.153.1:80. The second router's IP is 192.168.10.101:8080 is port forwarded to the second PVS and naturally, that PVS has the same IP 172.27.153.1:80. The two systems have been working fine for over a year with the integration until about a month or so ago. The only thing that changed are a couple of HA version upgrades.

Mini router details: https://www.gl-inet.com/products/gl-mt300n-v2/ I hope this helps. Let me know if I can be of further assistance.

DeviceList_PVS_1.txt DeviceList_PVS_2.txt

Screenshot of the SunPower integration: image

krbaker commented 1 day ago

Apologies this thread got dropped. Is this still an issue?

This integration won't be able to solve a 403 (if its a 403 for curl, it'll be a 403 for the integration)

At this point I don't think I've seen similar reports so I don't think its caused by a newer firmware or other people would be seeing it.

I would really suspect something along the path isn't behaving correctly (not HA, nor the PVS). To debug I would tcpdump the lan between the router and the PVS management interface and ensure 1) there is traffic there, 2) the PVS is really responding with a 403.