Plugin failing to load after upgrading to OpenSense 24.7

[craigwitter]

It looks like the widget rewrite has removed the www/widgets/api/ folder, so the addon is no longer able to find system.inc, and temperature.inc, when connecting to OpnSense 24.7.

[anipad]

Can confirm after todays upgrade:

`[25-Jul-2024 18:28:00 Etc/UTC] PHP Fatal error: Uncaught Error: Failed opening required '/usr/local/www/widgets/api/plugins/system.inc' (include_path='/usr/local/etc/inc:/usr/local/www:/usr/local/opnsense/mvc:/usr/local/opnsense/contrib:/usr/local/share/pear:/usr/local/share') in /usr/local/etc/inc/xmlrpc/hass.inc(12) : eval()'d code:5 Stack trace:

0 /usr/local/etc/inc/xmlrpc/hass.inc(12): eval()

1 /usr/local/opnsense/contrib/IXR/IXR_Library.php(446): exec_php_xmlrpc('\nini_set('displ...')

2 /usr/local/opnsense/contrib/IXR/IXR_Library.php(384): IXR_Server->call('opnsense.exec_p...', '\nini_set('displ...')

3 /usr/local/opnsense/contrib/IXR/IXR_Library.php(357): IXR_Server->serve('<?xml version='...')

4 /usr/local/etc/inc/xmlrpc.inc(67): IXR_Server->__construct(Array)

5 /usr/local/www/xmlrpc.php(104): XMLRPCServer->start()

6 {main}

thrown in /usr/local/etc/inc/xmlrpc/hass.inc(12) : eval()'d code on line 5`

[FredericMa]

Same here. This is the error in HA:

Unexpected error fetching Router OPNsense state data
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 312, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 268, in _async_update_data
    return await self.update_method()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/__init__.py", line 97, in async_update_data
    await hass.async_add_executor_job(lambda: data.update())
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/__init__.py", line 97, in <lambda>
    await hass.async_add_executor_job(lambda: data.update())
                                              ^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/__init__.py", line 316, in update
    self._state["telemetry"] = self._get_telemetry()
                               ^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/__init__.py", line 225, in inner
    response = func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/__init__.py", line 249, in _get_telemetry
    return self._client.get_telemetry()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/pyopnsense/__init__.py", line 101, in inner
    raise err
  File "/config/custom_components/opnsense/pyopnsense/__init__.py", line 98, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/pyopnsense/__init__.py", line 1037, in get_telemetry
    data = self._exec_php(script)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/pyopnsense/__init__.py", line 88, in inner
    response = func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opnsense/pyopnsense/__init__.py", line 131, in _exec_php
    response = self._get_proxy().opnsense.exec_php(script)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1122, in __call__
    return self.__send(self.__name, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1461, in __request
    response = self.__transport.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1166, in request
    return self.single_request(host, handler, request_body, verbose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1196, in single_request
    raise ProtocolError(
xmlrpc.client.ProtocolError: <ProtocolError for *ommited_sensitive_data@ip_address*/xmlrpc.php: 500 Internal Server Error>

[ricargr]

Same problem here

[ghammer80]

Same here, broke after the 24.7 update. Plug-in crashes in opnsense

[rama31244]

Yep same here, doesn't work straight after upgrade to 24.7. Is it an issue to leave it as is until the plugin is updated? Or should I downgrade Opnsense?

[spaghiari]

Hello i have same problem on my opnsense

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 312, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 268, in _async_update_data return await self.update_method() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/init.py", line 97, in async_update_data await hass.async_add_executor_job(lambda: data.update()) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/init.py", line 97, in await hass.async_add_executor_job(lambda: data.update()) ^^^^^^^^^^^^^ File "/config/custom_components/opnsense/init.py", line 316, in update self._state["telemetry"] = self._get_telemetry() ^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/init.py", line 225, in inner response = func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/init.py", line 249, in _get_telemetry return self._client.get_telemetry() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/pyopnsense/init.py", line 101, in inner raise err File "/config/custom_components/opnsense/pyopnsense/init.py", line 98, in inner return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/pyopnsense/init.py", line 1037, in get_telemetry data = self._exec_php(script) ^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/pyopnsense/init.py", line 88, in inner response = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/opnsense/pyopnsense/init.py", line 131, in _exec_php response = self._get_proxy().opnsense.exec_php(script) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1122, in call return self.send(self.name, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1461, in request response = self.transport.request( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1166, in request return self.single_request(host, handler, request_body, verbose) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/xmlrpc/client.py", line 1196, in single_request raise ProtocolError( xmlrpc.client.ProtocolError: <ProtocolError for /xmlrpc.php: 500 Internal Server Error>

[kayaozalp]

Same problem after 24.7 update.

[dwkirw]

crickets

[craigwitter]

If you edit the file /config/custom_components/opnsense/pyopnsense/init.py

Change line 858 and 859 to be include_once instead of require_once

and comment out the following lines (add // in the front of the line) 922-923, 946-965,969,971-975, 978-986 init.py.txt

The addon will connect, you won't have opensense system data (CPU, memory usage, etc), but you'll still get interface, gateway, etc.. I tried to rewrite part of it to use the new api's but wasn't successful. Hopefully this helps someone. I've attached a copy of the file here to make it easier too. If you want to use the file, you have to get rid of the txt extension as I couldn't upload that on the comment.

[heyitsyang]

Thanks @craigwitter. This got my WiFi presence back online.

[se7entynine]

Same problem with the latest OPNsense updates. Is there any update regarding this problem without the manual fix?

[Fannangir]

Same happens here. The manual fix still keeps all the errors. Is there a manual way to remove all the errors in the meanwhile?

[jasonpstokes]

The manual fix still keeps all the errors.

You shouldn't get errors if you edit or overwrite the pyopnsense \__init__.py file per Craig's post above (https://github.com/travisghansen/hass-opnsense/issues/143#issuecomment-2258885338) and then restart HA.

What did you do?

[Fannangir]

Interesting. Thanks for the tip. At first, I have commented the lines, which didn't help. After replacing the file, the error is solved, replaced by a warning below. I guess this is how it should work at the moment with the bypass?

This error originated from a custom integration.

Logger: py.warnings Source: custom_components/opnsense/init.py:49 integration: OPNsense (documentation, issues) First occurred: 19:26:31 (7 occurrences) Last logged: 19:26:31

/config/custom_components/opnsense/pyopnsense/init.py:801: SyntaxWarning: invalid escape sequence '\C' script = """ /config/custom_components/opnsense/pyopnsense/init.py:857: SyntaxWarning: invalid escape sequence '\D' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1046: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1072: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1145: SyntaxWarning: invalid escape sequence '\O' script = """

[craigwitter]

I suspect something was lost in encoding. Try retyping the end of each of those line numbers. They should be three quote marks " without spaces.

On Mon, Aug 5, 2024 at 12:31 PM Fannangir @.***> wrote:

Interesting. Thanks for the tip. At first, I have commented the lines, which didn't help. After replacing the file, the error is solved, replaced by a warning below. I guess this is how it should work at the moment with the bypass?

This error originated from a custom integration.

Logger: py.warnings Source: custom_components/opnsense/init.py:49 integration: OPNsense (documentation, issues) First occurred: 19:26:31 (7 occurrences) Last logged: 19:26:31

/config/custom_components/opnsense/pyopnsense/init.py:801: SyntaxWarning: invalid escape sequence '\C' script = """ /config/custom_components/opnsense/pyopnsense/init.py:857: SyntaxWarning: invalid escape sequence '\D' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1046: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1072: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1145: SyntaxWarning: invalid escape sequence '\O' script = """

— Reply to this email directly, view it on GitHub https://github.com/travisghansen/hass-opnsense/issues/143#issuecomment-2269466887, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADDNYG27WOCUWHTC4T4HLN3ZP6SFNAVCNFSM6AAAAABLO232U2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRZGQ3DMOBYG4 . You are receiving this because you were mentioned.Message ID: @.***>

[Fannangir]

Jut tried, same result. A snapshot of line 801 as an example:

Any guess what could be wrong?

[wormuths]

I'm holding off my 24.7 update until this is fixed. Following...

[KyleStilkey]

Interesting. Thanks for the tip. At first, I have commented the lines, which didn't help. After replacing the file, the error is solved, replaced by a warning below. I guess this is how it should work at the moment with the bypass?

This error originated from a custom integration.

Logger: py.warnings Source: custom_components/opnsense/init.py:49 integration: OPNsense (documentation, issues) First occurred: 19:26:31 (7 occurrences) Last logged: 19:26:31

/config/custom_components/opnsense/pyopnsense/init.py:801: SyntaxWarning: invalid escape sequence '\C' script = """ /config/custom_components/opnsense/pyopnsense/init.py:857: SyntaxWarning: invalid escape sequence '\D' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1046: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1072: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1145: SyntaxWarning: invalid escape sequence '\O' script = """

Same issue too and I even used your uploaded init.py just incase I messed something up. I'm going to guess the new Home Assistant 2024.8.0 update has made changes to Python which doesn't like those lines, just my guess but hoping someone with a bit more knowledge can take a look.

[Bubbalax]

I am holding off until a fix. Is there one being worked on?

[wormuths]

Same question... Any chance this is going to be fixed? I rely heavily on this integration...

[travisghansen]

If anyone knows the replacement functionality then a PR would be welcome, or even just pointing where the new code resides. Otherwise I will try to take a look in the next week or so.

[ashoktvm]

include_once

changed like this, But still the same

[NebKi]

include_once

changed like this, But still the same

Interesting. Thanks for the tip. At first, I have commented the lines, which didn't help. After replacing the file, the error is solved, replaced by a warning below. I guess this is how it should work at the moment with the bypass? This error originated from a custom integration. Logger: py.warnings Source: custom_components/opnsense/init.py:49 integration: OPNsense (documentation, issues) First occurred: 19:26:31 (7 occurrences) Last logged: 19:26:31 /config/custom_components/opnsense/pyopnsense/init.py:801: SyntaxWarning: invalid escape sequence '\C' script = """ /config/custom_components/opnsense/pyopnsense/init.py:857: SyntaxWarning: invalid escape sequence '\D' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1046: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1072: SyntaxWarning: invalid escape sequence '\O' script = """ /config/custom_components/opnsense/pyopnsense/init.py:1145: SyntaxWarning: invalid escape sequence '\O' script = """

Same issue too and I even used your uploaded init.py just incase I messed something up. I'm going to guess the new Home Assistant 2024.8.0 update has made changes to Python which doesn't like those lines, just my guess but hoping someone with a bit more knowledge can take a look.

Try redownloading the integration from HACS and then only replace the init.py file in the folder of ./custom_components/opnsense/pyopnsense/ with the one uploaded in this thread. There is an init.py in the main directory as well, but that is not the one that needs to be replaced.

I'm not getting any errors in OPNsense from this.

[vk2him]

For those saying the suggested fix doesn't work, I'd guarantee you're editing/replacing the wrong init.py file.

As NebKi mentions above, you have to edit/replace the init.py in ./custom_components/opnsense/pyopnsense/ and NOT the init.py in ./custom_components/opnsense/

The temporary workaround works fine on my HA

[PatrickWindegger]

Same problem - Workaround with init.py file not working for me.

Home Assistant Core 2024.7.4 OPNsense 24.7

Waiting for updates...

[NebKi]

Same problem - Workaround with init.py file not working for me.

Home Assistant Core 2024.7.4 OPNsense 24.7

Waiting for updates...

Are you saving the file uploaded in this thread as __init__.py and uploading it as such in the correct folder ./custom_components/opnsense/pyopnsense/, overwriting the existing one? And then restarted it after?

[PatrickWindegger]

Same problem - Workaround with init.py file not working for me. Home Assistant Core 2024.7.4 OPNsense 24.7 Waiting for updates...

Are you saving the file uploaded in this thread as __init__.py and uploading it as such in the correct folder ./custom_components/opnsense/pyopnsense/, overwriting the existing one? And then restarted it after?

I have edited/comment out the lines as described.

Now I have just downloaded the file, renamed it to “init.py” and uploaded it to the pyopnsense folder. Unfortunately without success. I get the PHP error “500 Internal Server Error”.

[NebKi]

Same problem - Workaround with init.py file not working for me. Home Assistant Core 2024.7.4 OPNsense 24.7 Waiting for updates...

Are you saving the file uploaded in this thread as __init__.py and uploading it as such in the correct folder ./custom_components/opnsense/pyopnsense/, overwriting the existing one? And then restarted it after?

I have edited/comment out the lines as described.

Now I have just downloaded the file, renamed it to “init.py” and uploaded it to the pyopnsense folder. Unfortunately without success. I get the PHP error “500 Internal Server Error”.

Sorry, I'm being dense. But Github appears to remove the underscores. Did you rename it to __init__.py or just init.py?

[PatrickWindegger]

Same problem - Workaround with init.py file not working for me. Home Assistant Core 2024.7.4 OPNsense 24.7 Waiting for updates...

Are you saving the file uploaded in this thread as __init__.py and uploading it as such in the correct folder ./custom_components/opnsense/pyopnsense/, overwriting the existing one? And then restarted it after?

I have edited/comment out the lines as described. Now I have just downloaded the file, renamed it to “init.py” and uploaded it to the pyopnsense folder. Unfortunately without success. I get the PHP error “500 Internal Server Error”.

Sorry, I'm being dense. But Github appears to remove the underscores. Did you rename it to __init__.py or just init.py?

I renamed it to __init__.py with underscores. ;-)

[PatrickWindegger]

...save ...the file uploaded in this thread as __init__.py

...I have just downloaded the file, renamed it to init.py

If this is what you did, can you see the difference in your file name to the one you quoted?

You have to save the downloaded file name with underscores into the custom_components > opnsense > pyopnsense folder. As others have said: don't do anything to the file with the same name in the first opnsense folder - keep going!

Sorry - I renamed it to __init__.py Not sure if I forgot the underscores in my comment or if github removed them.

[BatSmacker84]

If you edit the file /config/custom_components/opnsense/pyopnsense/init.py

Change line 858 and 859 to be include_once instead of require_once

and comment out the following lines (add // in the front of the line) 922-923, 946-965,969,971-975, 978-986 init.py.txt

The addon will connect, you won't have opensense system data (CPU, memory usage, etc), but you'll still get interface, gateway, etc.. I tried to rewrite part of it to use the new api's but wasn't successful. Hopefully this helps someone. I've attached a copy of the file here to make it easier too. If you want to use the file, you have to get rid of the txt extension as I couldn't upload that on the comment.

Something I noticed with these instructions that seems to be incorrect is that changing line 860 to be include_once is not stated. I'm guessing that the mention of line 859 is supposed to be 860 because 859 is already include_once.

The provided file and proposed PR already make this change, but if someone were to follow the written instructions (me) then it won't work.

[beren12]

Thanks for the info to get it partly working!

[alexdelprete]

If anyone knows the replacement functionality then a PR would be welcome, or even just pointing where the new code resides. Otherwise I will try to take a look in the next week or so.

From a first look, the data we needed is available via the Diagnostics API now (SystemController.php and FirewallController.php specifically). the new widgets pull info via API too: https://docs.opnsense.org/development/api/core/diagnostics.html

[cmerkle]

Is any fix for this in the works? Having the same issue. I was able to get the interface statistics working by modifying the init.py as mentioned above but still hoping for a full fledged fix

[jasonpstokes]

Is any fix for this in the works? Having the same issue. I was able to get the interface statistics working by modifying the init.py as mentioned above but still hoping for a full fledged fix

Sounds promising - please share what you did to get that working, as it might speed up the fix...

[vk2him]

Is any fix for this in the works? Having the same issue. I was able to get the interface statistics working by modifying the init.py as mentioned above but still hoping for a full fledged fix

Sounds promising - please share what you did to get that working, as it might speed up the fix...

He following the instructions above ^^^ and replaced the __init__.py https://github.com/travisghansen/hass-opnsense/issues/143#issuecomment-2258885338

[jasonpstokes]

He following the instructions above ^^^ and replaced the __init__.py #143 (comment)

Oh my bad - I read it like cmerkle had something working using the Diagnostics API.

[buzzard10]

Is it a problem in OPNsense? Or in HASS? I have issue, i can enter credentials to OPNsense inside HASS, in integration, but then i receive an error..

[baudneo]

In the opnsense have add-on. It doesn't support the new diagnostic API and the old API method has been removed.

[buzzard10]

So what's the way? I should downgrade opnsense? Or wait for upgrade? Is there any deadline for the fix?

[daernsinstantfortress]

So what's the way? I should downgrade opnsense? Or wait for upgrade? Is there any deadline for the fix?

Copy this file as described and it will restore most functionality. If you need the diagnostic sensors that have been removed, you'll need to rollback OPNSense.

[lindem7804]

afer putting the file in folder as described above & restartet HA, then the plugin works for me again....thank you

[VossiVersuchtsMal]

@craigwitter works like a charm Thank you.

[buzzard10]

@daernsinstantfortress Hmm, i have this in my directory: Do you want me to comment lines? Or you want me to paste content of this file into it?

[Jelsie2]

@buzzard10 in the last line he says that if you want to use the file, you will need to remove the .txt extension. Worked for me like a charm. But if I would have known this, I would not have upgraded.

[buzzard10]

I have this error: Nothing in logs :(

I have the newest version of opnsense and home assistant. Opnsense is new, because i just set this up.

[alexdelprete]

v0.2.0-beta.1 has been released. Please test it and give us a feedback.

I created a thread in the Discussions section, let's continue there, hope to receive other positive/negative feedbacks.

I'll close this issue.