Custom integration won't load with 2023.9 & 2023.9.1

[jamespreedy]

Describe the bug The integration worked fine immediately after 2023.9 was released on Wednesday. On Thursday I noticed that my Solax device went dead and all my entities went unavailable. Rebooting the host brought it back to life. 2023.9.1 came out yesterday and the integration now won't load at all.

Mandatory details

1. Integration Version: 2023.9.3
2. HA Version: 2023.9.1
3. Inverter Brand: SolaX
4. Plugin Used: SolaX
5. First 6 digits of Serial: H34A10
6. Connection Method: Waveshare RS485 to Eth

Detailed Error Log

Logger: homeassistant.setup
Source: loader.py:826 
First occurred: 5:56:58 am (1 occurrences) 
Last logged: 5:56:58 am

Setup failed for custom integration solax_modbus: Unable to import component: Exception importing custom_components.solax_modbus
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/config/custom_components/solax_modbus/__init__.py", line 31, in <module>
    from pymodbus.payload import BinaryPayloadBuilder, BinaryPayloadDecoder, Endian
  File "/usr/local/lib/python3.11/site-packages/pymodbus/payload.py", line 28, in <module>
    class BinaryPayloadBuilder:
  File "/usr/local/lib/python3.11/site-packages/pymodbus/payload.py", line 42, in BinaryPayloadBuilder
    self, payload=None, byteorder=Endian.Little, wordorder=Endian.Big, repack=False
                                  ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/enum.py", line 784, in __getattr__
    raise AttributeError(name) from None
AttributeError: Little

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 215, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 826, in get_component
    raise ImportError(f"Exception importing {self.pkg_path}") from err
ImportError: Exception importing custom_components.solax_modbus

Additional context This custom component has been flawless since setting it up. Nothing I'm aware of has changed from a physical install perspective, and the Waveshare still has the blue connected light on it.

I believe there were a bunch of Modbus changes to HA 2023.9?

I also have a Nilan MVHR which also connects via Modbus (same Waveshare RS485 to Eth) - the custom integration for this also went dead following the 2023.9 HA update.

Appreciate this might be broken on the HA side, but just wanted to make sure I was reporting it. If there is any further information I can get for you to help resolve this, please let me know. 🍺

[infradom]

I did an upgrade on my system to 2023.9.1, it went without problems. I did not go through the setup procedure though. I will also remove the integration and redo the setup to see whether the error is in that area. I have no other integration that uses modbus, this might explain the difference.

[mstefany]

Happens in my case as well. It might be related to this change? https://github.com/pymodbus-dev/pymodbus/pull/1743

[wills106]

Just released a new Beta, renamed all the Big / Little to BIG/LITTLE also updated pymodbus to 3.5.1 a per HA 2023.9.x

2023.09.4b1 could someone try it and let me know?

[eklofmar]

I just updated to the beta and restarted the system. I still get the same error.

[wills106]

I was on HA 2023.8.4 and this beta worked fine, just gone to 2023.9.1 and wont even read my serial. But no errors either...

[eklofmar]

On HA 2023.8.4 it worked fine.

After updated to the beta I got reading error. And after system restart it wont load.

[infradom]

New version gives me this error: Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/solax_modbus/init.py", line 109, in async_setup_entry plugin = importlib.importmodule(f".plugin{plugin_name}", 'custom_components.solax_modbus') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1204, in _gcd_import File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 690, in _load_unlocked File "", line 940, in exec_module File "", line 241, in _call_with_frames_removed File "/config/custom_components/solax_modbus/plugin_solax.py", line 5389, in order16 = Endian.BIG, ^^^^^^^^^^ AttributeError: type object 'Endian' has no attribute 'BIG'

[wills106]

@infradom is your pyModbus on less than 3.5.x ? I have updated mine to 3.5.1 and it doesn't complain it has no attribute.

It just can't read the serial and gives no feedback.

[wills106]

Just so people know this isn't fixed yet and might take a day or two as there isn't anything obviously wrong.

If you don't use the HA Modbus for anything then HA 2023.9.1 and Integration 2023.09.3 works ok as it downgrades pyModbus to 3.3.1.

If you need to use the HA Modbus in addition to this Integration you need to stay on HA 2023.8.x

[wartungsmonteur]

Guten Morgen

Ich hatte das selbe Problem. Lösung war der komplette Neustart des Home-Assistenten und ganz untenvheraus und ein Neustart des waveshare Moduls, also dieses stromlos machen und neu starten. Nach dem Neustart muss eventuell das Addon händisch neu geladen lassen.

[uspino2]

Same thing here (not working), my errors are:

Growatt_Modbus: reading firmware number from address 0x9 failed; other address may succeed
Growatt_Modbus: reading firmware number from address 0xbb9 failed; other address may succeed
Growatt_Modbus: cannot find firmware version, even not for other Inverter
pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] Failed to connect[ModbusTcpClient(10.10.10.126:502)]

Thanks.

[mudshurko]

same problem since Sept.07 resetted the system today to : Home Assistant 2023.8.4 Supervisor 2023.08.3 Operating System 10.5 Frontend 20230802.1 - latest and installed Solax Inverter Modbus completly new no effect 1 device, 4 entities.... I hope wills can find the problem... lg

[wartungsmonteur]

Das Addon ist ok. Man muss die neueste Version vom HA installieren, dann Solax via HACS updaten und das komplette System neu starten. Ich musste zudem die waveshare Module kurz vom Stromnetz trennen und neu booten lassen. Danach ging alles wieder ohne Probleme.

[uspino2]

Das Addon ist ok. Man muss die neueste Version vom HA installieren, dann Solax via HACS updaten und das komplette System neu starten. Ich musste zudem die waveshare Module kurz vom Stromnetz trennen und neu booten lassen. Danach ging alles wieder ohne Probleme.

Restarting, power cycle, etc did not work me

[wartungsmonteur]

Hast du auch mal die waveshare stromlos gemacht ?

[uspino2]

Hast du auch mal die waveshare stromlos gemacht ? Have you ever de-energised the waveshare?

Yes, left it unplugged for a few minutes

[wartungsmonteur]

Welche LED Leuchten ? 1 x Rot und 2 x Blau ? Wenn ja, das Addon neu laden. Ich hatte auch dies Probleme. Bestimmt fünf Mal neu gestartet und irgendwann ging es plötzlich.

[mudshurko]

fine when the system with the waveshare is working (what is this waveshare exactly?) I get data from the Solax X3 from COM port, using an USB - RS485 Konverter the raspi is a Raspberry Pi 3 and this doesnt work now yesterday i set up a new system, new image on a new SD-card, but nothing changed it looks like a blocked Modbus transfer HA gets no entities and no data now I'm back on the "old" SD card, re-started for many times, no effect

[wartungsmonteur]

Hallo Ich nutze diesen waveshare

[mudshurko]

hmmm this is serial to TCP not USB - RS485 maybe the problem occurs in /dev/ttyUSB0 ?

[uspino2]

HA just released 2023.9.2 but the problem is still there. Release notes say the update includes

Bump pymodbus to v3.5.2
Bump version, due to a connect problem, that is causing problems for the modbus integration

Hopefully this helps find the issue with Solax integration. Any update on the work @wills106 ? Thanks.

[filipi992062]

I updated to Home assistant 2023.9.2 this morning and the problem was solved.

[uspino2]

I updated to Home assistant 2023.9.2 this morning and the problem was solved.

@filipi992062 Are you on Solax 2023.09.3 or 2023.09.4b1?

[wills106]

Hopefully this helps find the issue with Solax integration.

Updating pyModbus to 3.5.2 with Integration 2023.09.4b1 didn't help. When you add the Integration it fails to find a serial Number and doesn't give any feedback as to what's failed. I'll need to add in more debugging when I can.

I updated to Home assistant 2023.9.2 this morning and the problem was solved.

Didn't get chance to update to 2023.9.2 this morning. If you are on Integration 2023.09.3 and not using the built in Modbus component the Integration could have downgraded pyModbus to 3.3.1

If you use pip list in your HA container it will list all the python modules and their versions.

[jamespreedy]

Not sure what help I am here, but perhaps just as FYI. I updated to HA 2023.9.2 this morning too. I'm still running SolaX 2023.9.3.

I don't use the native HA modbus component (modbus: isn't in my configuration.yaml)

I get the below error in my logs.

Logger: homeassistant.loader
Source: custom_components/solax_modbus/__init__.py:31 
Integration: solax_modbus (documentation, issues) 
First occurred: 7:06:14 am (1 occurrences) 
Last logged: 7:06:14 am

Unexpected exception importing component custom_components.solax_modbus
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/config/custom_components/solax_modbus/__init__.py", line 31, in <module>
    from pymodbus.payload import BinaryPayloadBuilder, BinaryPayloadDecoder, Endian
  File "/usr/local/lib/python3.11/site-packages/pymodbus/payload.py", line 28, in <module>
    class BinaryPayloadBuilder:
  File "/usr/local/lib/python3.11/site-packages/pymodbus/payload.py", line 42, in BinaryPayloadBuilder
    self, payload=None, byteorder=Endian.Little, wordorder=Endian.Big, repack=False
                                  ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/enum.py", line 784, in __getattr__
    raise AttributeError(name) from None
AttributeError: Little

and I also get

Source: loader.py:826 
First occurred: 7:06:14 am (1 occurrences) 
Last logged: 7:06:14 am

Setup failed for custom integration solax_modbus: Unable to import component: Exception importing custom_components.solax_modbus
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/config/custom_components/solax_modbus/__init__.py", line 31, in <module>
    from pymodbus.payload import BinaryPayloadBuilder, BinaryPayloadDecoder, Endian
  File "/usr/local/lib/python3.11/site-packages/pymodbus/payload.py", line 28, in <module>
    class BinaryPayloadBuilder:
  File "/usr/local/lib/python3.11/site-packages/pymodbus/payload.py", line 42, in BinaryPayloadBuilder
    self, payload=None, byteorder=Endian.Little, wordorder=Endian.Big, repack=False
                                  ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/enum.py", line 784, in __getattr__
    raise AttributeError(name) from None
AttributeError: Little

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 215, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 826, in get_component
    raise ImportError(f"Exception importing {self.pkg_path}") from err
ImportError: Exception importing custom_components.solax_modbus

I also use modbus via another custom component (https://github.com/veista/nilan) for my Nilan MVHR. Its still not working since me upgrading to HA 2023.9.x, with pretty similar symptoms.

Given that I don't use native HA modbus, and just these custom components, which both died at the same time, I'm not thinking it's a coincidence ... 🤞🏻

@wills106 - as always, thanks so much of your ongoing effort and support with this custom component - really appreciate it! 🍺

[PutoPunko]

Hi,

Updated to HA 2023.9.2 with SolaX 2023.09.4b1 (connected to inverter directly to ETH port) and works OK (I am using as well HA modbus integration for other devices).

Best regards,

[uspino2]

For me, HA 2023.9.2 works with Solax 2023.9.3, but not with 2023.9.4b1...

[uspino2]

If you use pip list in your HA container it will list all the python modules and their versions.

My Home Assistant is a VM (Hyper-X). How can I check modules version?

[alienatedsec]

If you use pip list in your HA container it will list all the python modules and their versions.

My Home Assistant is a VM (Hyper-X). How can I check modules version?

@uspino2 you need to install the Advanced SSH & Web Terminal addon and can do anything as you would via normal putty connection

[PutoPunko]

Hi,

Updated to HA 2023.9.2 with SolaX 2023.09.4b1 (connected to inverter directly to ETH port) and works OK (I am using as well HA modbus integration for other devices).

Best regards,

[uspino2]

If you use pip list in your HA container it will list all the python modules and their versions.

My Home Assistant is a VM (Hyper-X). How can I check modules version?

@uspino2 you need to install the Advanced SSH & Web Terminal addon and can do anything as you would via normal putty connection

Thanks, I tried that, but I get:

[alienatedsec]

@uspino2 mine works fine

[wills106]

It's a bit more complicated on HAOS and I haven't used it in ages, but you need to check in the equivalent of the core container. There should be a lot more python packages showing like:

[wills106]

Just released 2023.09.4b2 I think I have found why 2023.09.b1 was failing, there was another change in pyModbus 3.4.0.

I can confirm that forcing pyModbus to 3.5.2 on HA 2023.9.2 but not using the built in HA Modbus is working for me on 2023.09.4b2. The likes of @jamespreedy @mstefany are probably best confirming before I release a stable version as it looks they they are using this Integration in addition to the built in HA Modbus?

[jamespreedy]

@wills106 - hey mate, thank you for all you help! We all owe you a beer for this one lol.

Just reading your comment above - I don't understand this all at the best of times, but I don't think that I'm using built in HA Modbus? I don't have "modbus: " in my configuration.yaml - is that what you meant?

[jamespreedy]

On that note, I've just pulled and tried beta 2 - still getting the same error I mentioned in this other bug - https://github.com/wills106/homeassistant-solax-modbus/issues/530. Think I've broken something by uninstalling and re-installing. 🤦🏻‍♂️🤦🏻‍♂️🤦🏻‍♂️

Aside from removing the integration from HACS, and making sure there's nothing in the custom_components folder, is there a more complete way of clearing the solax integration and trying again?

[wills106]

Sorry I misread thinking your Nilan MVHR was using the built in HA Modbus, but I just reread it and you mentioned using another Integration for it.

[wills106]

I think the best way would be to:

Remove SolaX Modbus Integration from Integration page. Remove SolaX Modbus from HACS. Ensure solax_modbus folder is no longer in custom_components Restart HA Download SolaX through HACS Restart HA Add SolaX Modbus on the Integration Page.

[wills106]

For anyone trying Integration 2023.09.4b2, there is two potential errors I have not check for if you are still on pyModbus 3.3.1 or earlier:

'slave=slave' or 'Endian.BIG' / 'Endian.LITTLE'

I might need to drop legacy support for pyModbus and HA as we will potentially have 3 main variations: pyModbus 2.5.3, pyModbus 3.0.x-3.3.1 pyModbus 3.5.x and above

I'll perform testing on Integration 2023.09.4b2 and pyModbus 3.3.1 in the next day or two unless anyone reports that combination doesn't work.

It still doesn't address why #530 is happening though.

[wartungsmonteur]

Betrifft dieser Fehler nur die USB Variante ? Per TCP funktioniert es ohne Probleme.

[wills106]

It can affect both, the Integration 2023.09.3 can downgrade your pyModbus to 3.3.1 even on HA 2023.9.x so it appears you don't have an issue.

But if your HA 2023.9.x forces pyModbus to 3.5.x that's when you have this issue.

So for example if you use the built in HA Modbus it requires 3.5.x on HA 2023.9.x and above. So using Integration 2023.09.3 becomes an issue. That's what Integration 2023.09.4b2 is trying to fix, getting all versions to cooperate.

[filipi992062]

I updated to Home assistant 2023.9.2 this morning and the problem was solved.

@filipi992062 Are you on Solax 2023.09.3 or 2023.09.4b1?

Solax 2023.09.4b1

[Matz88]

Also after upgrading to 2023.09.4b2 I unfortunately still have the same issue:

File "/config/custom_components/solax_modbus/plugin_sofar.py", line 2430, in order16 = Endian.Big, ^^^^^^^^^^

[wills106]

Have you tried: Removing the Integration from the Integration page Upgrading to 2023.09.4b2 Restarting HA Re-adding the Integration?

[Matz88]

being it already 2023.09.4b2, shall I first downgrade?

[Matz88]

Tried with downgrade first, still some error, no entities loaded

[wills106]

Just released 2023.09.4b3 which pins pyModbus to 3.5.2, it should help with people stuck on 3.3.1 etc.

[Matz88]

Tried the beta 3 but still I have the same error:

File "/config/custom_components/solax_modbus/plugin_sofar.py", line 2430, in order16 = Endian.Big, ^^^^^^^^^^

Is it still the same issue or something different? I have also tried an HW reboot after upgrading. What else can I try?

[wills106]

order16 = Endian.Big,

I have found the error, I didn't convert all of those to Endian.BIG

Will push another update out.

[Matz88]

Cool! Many thanks!