Binary sensor in ONVIF integration not available after last update with Bosch FLEXIDOME IP turret 3000i IR

[lucabress]

The problem

Hi after the last update 2023.4, the binary sensors that are part of the ONVIF integration of my IP cams (Bosch) are no longer available (digital input, really triggered, motion detection)

What version of Home Assistant Core has the issue?

core-2023.4.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ONVIF

Link to integration documentation on our website

https://www.home-assistant.io/integrations/onvif/

Diagnostics information

config_entry-onvif-1b56b8f1e3dba44312f93d9deec90f3b.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @hunterjm, mind taking a look at this issue as it has been labeled with an integration (onvif) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `onvif` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign onvif` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

onvif documentation onvif source _{^{(message by IssueLinks)}}

[bdraco]

Please post debug logs using the troubleshooting steps here

https://www.home-assistant.io/docs/configuration/troubleshooting/#debug-logs-and-diagnostics

[lucabress]

@bdraco I hope i did everything right...

config_entry-onvif-1b56b8f1e3dba44312f93d9deec90f3b.json.txt home-assistant_onvif_2023-04-09T20-28-08.768Z.log

[bdraco]

2023-04-09 19:02:07.545 ERROR (MainThread) [zeep.xsd.types.simple] Error during xml -> python translation
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/zeep/xsd/types/simple.py", line 79, in parse_xmlelement
    return self.pythonvalue(xmlelement.text)
  File "/usr/local/lib/python3.10/site-packages/zeep/xsd/types/builtins.py", line 44, in _wrapper
    return func(self, re.sub(r"[\n\r\t ]", " ", value).strip())
  File "/usr/local/lib/python3.10/site-packages/zeep/xsd/types/builtins.py", line 180, in pythonvalue
    return isodate.parse_datetime(value)
  File "/usr/local/lib/python3.10/site-packages/isodate/isodatetime.py", line 56, in parse_datetime
    tmptime = parse_time(timestring)
  File "/usr/local/lib/python3.10/site-packages/isodate/isotime.py", line 148, in parse_time
    raise ISO8601Error('Unrecognised ISO 8601 time format: %r' % timestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 time format: '-07:03:03Z'
2023-04-09 19:02:07.547 ERROR (MainThread) [zeep.xsd.types.simple] Error during xml -> python translation
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/zeep/xsd/types/simple.py", line 79, in parse_xmlelement
    return self.pythonvalue(xmlelement.text)
  File "/usr/local/lib/python3.10/site-packages/zeep/xsd/types/builtins.py", line 44, in _wrapper
    return func(self, re.sub(r"[\n\r\t ]", " ", value).strip())
  File "/usr/local/lib/python3.10/site-packages/zeep/xsd/types/builtins.py", line 180, in pythonvalue
    return isodate.parse_datetime(value)
  File "/usr/local/lib/python3.10/site-packages/isodate/isodatetime.py", line 56, in parse_datetime
    tmptime = parse_time(timestring)
  File "/usr/local/lib/python3.10/site-packages/isodate/isotime.py", line 148, in parse_time
    raise ISO8601Error('Unrecognised ISO 8601 time format: %r' % timestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 time format: '-07:02:03Z'

[bdraco]

It looks like the camera is sending an invalid time stamp which is tripping up the XML parser

[bdraco]

https://github.com/mvantellingen/python-zeep/blob/377d9313b1b4807a31a5ee42227f1dc7e7e0471e/src/zeep/xsd/types/builtins.py#L180

[lucabress]

Ok, thanks, what can I do? I did various tests such as disabling and changing the NTP invert month and day I also updated the camera to the latest firmware but the problem remains...

[bdraco]

The expected datetime format is %Y-%m-%dT%H:%M:%S.%f%Z ex 2023-04-10T04:00:32Z

It looks like your camera sends the datetime as %Y-%m-%d-%H:%M:%S.%f%Z ex 2023-04-10-04:00:32Z

There is a - where it should be sending a T

This is likely a bug in the firmware of the camera.

[bdraco]

I opened a PR to zeep to handle this

https://github.com/mvantellingen/python-zeep/pull/1370

I have other PRs waiting so it might be a few months before they merge the workaround.

[bdraco]

I'd try to get the camera vendor to fix their firmware in the meantime

[bdraco]

Please try 2023.4.5

https://github.com/home-assistant/core/pull/91485 will add support for webhooks which might work around this problem if 2023.4.5 doesn't fix it

[lucabress]

Hi, sorry but I can only answer now, thanks for all the support and interest you are giving me, anyway even after the last update 2023.4.5 the entities still doesn't work...

[bdraco]

https://github.com/home-assistant/core/issues/37545#issuecomment-1513629724

[lucabress]

Tried but the problem remains :(

[bdraco]

I think we are going to have to wait for the vendor to fix the firmware or https://github.com/mvantellingen/python-zeep/pull/1370 to merge and release the workaround

[lucabress]

I think the vendor's firmware is ok from a certain point of view because everything worked until a month ago, I ran an update after finding the problem... hopefully the next updates will fix something, thanks again

[bdraco]

Can you downgrade the firmware to the previous version?

[lucabress]

i’ve just do the downgrade, it works… Ps downgrade to 2023.3.5

[bdraco]

mkdir -p /config/custom_components
cd /config/custom_components
git clone https://github.com/bdraco/onvif
cd /config/custom_components/onvif
git pull
git log -1

This version should work better with cameras with broken timezone handling.

[lucabress]

upgraded to 2023.4.6 and the problem reoccurred, performed the above but the problem remains...

[bdraco]

I'm not having any luck figuring out a solution here as the zeep logging doesn't give another trace data.

I ended up ordered the camera to see if I can come up with a workaround

[lucabress]

Ok, thank you! I am currently working with the 2023.3.5 and everything continues to work fine, as soon as I can I will install HA in my old raspberry, in order to comfortably try the next releases, if I can help you in any way let me know, thanks again for the support and the interest!

[bdraco]

There are some more fixes in 2023.5.x from other camera I received over the last few weeks. That might help, but if not the debug logs should give a lot more information

[lucabress]

Hi, I tried update 2023.5.0

The problem apparently seemed solved because it sees the "digital input" entity (the entity that interests me the most) but in reality it doesn't read it correctly, because it doesn't see the change from on to off

In addition to this, it asked to reconfigure my 2 Hikvision ipcams, I re-enter th e credentials but it still gives me an error, so the 2 ipcams remain unavailable 😩😩😩

Back to 2023.3.5 and all back to work

Attached are the debugs

config_entry-onvif-1b56b8f1e3dba44312f93d9deec90f3b.json.txt

home-assistant_onvif_2023-05-04T20-23-34.843Z.log

home-assistant_onvif_2023-05-04T20-24-44.768Z.log

[bdraco]

It looks like you still have the custom version in there

2023-05-04 22:21:23.526 WARNING (SyncWorker_2) [homeassistant.loader] We found a custom integration onvif which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-05-04 22:21:23.526 ERROR (SyncWorker_2) [homeassistant.loader] The custom integration 'onvif' does not have a version key in the manifest file and was blocked from loading. See https://developers.home-assistant.io/blog/2021/01/29/custom-integration-changes#versions for more details

Can you try taking it out and try again?

[lucabress]

Ok, how do I get it off like you say?

However, I tried again to delete the bosch camera and reintegrate it, the entities disappear as in previous updates

regarding hikvision continues authentication error

[bdraco]

rm -rf /config/custom_components/onvif

Restart

[lucabress]

I entered this line on the terminal and restarted but if I debug it tells me that I still have the custom component...

[bdraco]

Check to make sure there isn't another copy with another name and remove that as well

ls -l /config/custom_components

[lucabress]

Ok i'm freaking out please help me!

updated to 2023.5.1

I type: ls -l /config/custom_components

I see in the list : drwxr-xr-x 4 root root 4096 Apr 19 02:30 onvif

I type: rm -rf /config/custom_components/onvif

drwxr-xr-x 4 root root 4096 Apr 19 02:30 onvif disappears

if I debug I keep seeing "We found a custom integration onvif which has not been tested by Home Assistant" etc

[bdraco]

Do you by change have another directory where the custom component is installed?

find /config -name "*onvif*"

[lucabress]

[lucabress]

Ok, I confirm that the problem is given after the update 2023.4 and that the latest version where it works is 2023.3.5

I installed HA in my old raspberry 3B and with the virgin system acquiring the camera it only detects 6 entities instead of 16

Further problem from the 2023.5.0 update, the hikvision ipcams become unusable because they constantly give an authentication error

[bdraco]

Lets try increasing the logging so we can get a better idea whats going wrong

# Example configuration.yaml entry
logger:
  default: info
  logs:
    homeassistant.components.onvif: debug
    httpx: debug
    onvif: debug
    zeep: debug

Please add the above and restart

[lucabress]

Ok, done, what should I do once added?

[bdraco]

Restart, try to add the camera and than post the full log

[lucabress]

home-assistant_2023-05-06T22-54-39.389Z.log

[bdraco]

2023-05-07 00:31:54.522 DEBUG (MainThread) [onvif] /usr/local/lib/python3.10/site-packages/onvif/wsdl/events.wsdl: SetSynchronizationPoint failed

2023-05-07 00:31:54.509 DEBUG (MainThread) [httpx] HTTP Request: POST http://172.20.0.5/Web_Service?Idx=0 "HTTP/1.1 401 Unauthorized"

2023-05-07 00:38:57.846 DEBUG (MainThread) [homeassistant.components.onvif] Ingresso: Could not authenticate with camera: The action requested requires authorization and the sender is not authorized (code:env:Sender) (subcodes:{http://www.onvif.org/ver10/error}NotAuthorized)

[bdraco]

It looks like the user configured for the 172.20.0.5 camera isn't authorized to set a sync point.

[bdraco]

What does the integrations screen look like?

Are the cameras failing to setup or is it just the events not working?

[lucabress]

Problem with hikvision solved, don't ask me why, but initially I had acquired the hikvision cameras by putting the ip address and same password and username of the camera, from the 2023.5.0 update, you have to enter the camera menu go to the integration protocol menu, and create the onvif user

[MathiasSmedberg]

Why has the ONVIF integration been changed? My cameras used to work with "non compliant timestamp". Now the motion detection has been disabled in HA and if I delete the camera and add it again there isn't the same entities added? The pictures are from two identical cameras, nothing has changed int the firmware so it has to be a "HA problem".

[bdraco]

Unfortunately it’s a case of bad luck that we are hitting the bug in the cameras as we are using a few new endpoints in 2023.5.x in order to support more cameras.

[bdraco]

If you would like to help figure out which endpoint triggers the issue and are able to get a wire shark or tcpdump that would be helpful in developing a workaround for the problem

[MathiasSmedberg]

Unfortunately it’s a case of bad luck that we are hitting the bug in the cameras as we are using a few new endpoints in 2023.5.x in order to support more cameras.

Ok, I understand. Is it possible to install the old version of Onvif? All my cameras are "old" and im not planning to install any new cameras so I would not need the new features.

If you would like to help figure out which endpoint triggers the issue and are able to get a wire shark or tcpdump that would be helpful in developing a workaround for the problem

I would love to help but thats beyond my knowledge :( I have wireshark installed but never learned how to use it.

[bdraco]

You can copy the 2023.4.6 files (https://github.com/home-assistant/core/tree/2023.4.6/homeassistant/components/onvif) from 2023.4.x into /config/custom_components/onvif/

You'll need to add a version key to the manifest.json as well for it to be accepted and picked up.

That should work for a few versions. Hopefully by than we can get a wireshark dump, we get more information, or someone can contribute a solution, or zeep merges the time parser fix https://github.com/mvantellingen/python-zeep/pull/1370. I've tried to order the camera listed above a few times but haven't had any luck since its been discontinued (I keep getting orders canceled). I'm not sure if yours is the same model, but it would help to know which one in case someone still has an old one in stock.

[MathiasSmedberg]

You can copy the 2023.4.6 files (https://github.com/home-assistant/core/tree/2023.4.6/homeassistant/components/onvif) from 2023.4.x into /config/custom_components/onvif/

You'll need to add a version key to the manifest.json as well for it to be accepted and picked up.

That should work for a few versions. Hopefully by than we can get a wireshark dump, we get more information, or someone can contribute a solution, or zeep merges the time parser fix mvantellingen/python-zeep#1370. I've tried to order the camera listed above a few times but haven't had any luck since its been discontinued (I keep getting orders canceled). I'm not sure if yours is the same model, but it would help to know which one in case someone still has an old one in stock.

My Bosch camera is an old discontinued DINION NBN-498-P IVA (CPP3). The Axis cameras is still working.

I have access to Bosch cameras at work from time to time. If I learn Wireshark, if that´s what you need, maybe I could do some testing. EDIT: But no IP turret 3000i IR at work right now.

[bdraco]

My Bosch camera is an old discontinued DINION NBN-498-P IVA (CPP3). The Axis cameras is still working.

Thanks. I'm on a work trip right now for my day job, but I'll see if I can find one when I get back to the US.

I have access to Bosch cameras at work from time to time. If I learn Wireshark, if that´s what you need, maybe I could do some testing.

That would be great.

EDIT: But no IP turret 3000i IR at work right now.

[bdraco]

I was able get an old Bosch camera

Initial testing looks like my pr to zeep will fix the issue. Unfortunately nobody has looked at it yet

[bdraco]

It's a bit complex, but think we could work around the issue without waiting for zeep to merge my PR by making a copy of the datetime type and overriding the serialize function and than reregistering the type on the schema document to add support for the non conformant timestamp

https://github.com/mvantellingen/python-zeep/blob/main/src/zeep/xsd/schema.py