ispysoftware / iSpy

Open source surveillance software
http://www.ispyconnect.com
Other
1.35k stars 534 forks source link

ONVIF: error "The data in element 'Security' must be understood but cannot be handled" #177

Closed timmeh87 closed 2 years ago

timmeh87 commented 2 years ago

Hello, I am having trouble connecting to my off-brand v380 PTZ camera over onvif. I can control it from Homeassistant through their ONVIF integration so it surely works on my network already. I am running the docker container on a fresh ubuntu-18 host inside hyper-v (im NOT using docker desktop) and I have enabled --network host to get around any possible UDP problems. I think I was actually getting a different error before due to that, but now this is what it says, seems like a parsing error after connection is established:

ONVIF Discovery: Couldn't get device time - using system UTC time
Error: ONVIF Device Capabilities: ONVIF Device Capabilities: The data in element 'Security' must be understood but cannot be handled
Error: ONVIF Device Capabilities:    at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)
   at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.<>c__DisplayClass1_0.<CreateGenericTask>b__0(IAsyncResult asyncResult)
--- End of stack trace from previous location where exception was thrown ---
   at CoreLogic.Extensions.WithCancellation[T](Task`1 task, CancellationToken cancellationToken)
   at CoreLogic.Onvif.ONVIFDevice.GetDeviceCapabilities()
The data in element 'Security' must be understood but cannot be handled
Error: ONVIF Device: ONVIF Device: No media endpoints found
Error: ONVIF Device:    at CoreLogic.Onvif.ONVIFDevice.Connect()
No media endpoints found
Error: Failed to connect
Error:    at CoreLogic.Server.JsonProcessor.LoadJson(Int32 oid, Int32 ot, String lc, String command, String fullcmd, String data, RTCSession session)
Failed to connect

I have tried to use the "no profile" option which appears to search ALL ports and URLs, and it is always the same result, many "IP Camera" options but no "ONVIF" options

I will do more investigating and add anything I found here. I think there are some tools to use to sniff the traffic which Im going to look at when I have the time.

ispysoftware commented 2 years ago

from googling it looks like this is something to do with a failed SSL connection - if you want to port forward the camera and send me a login i can take a look at it from here

timmeh87 commented 2 years ago

Hello I have done further research into the problem. I found a python script to make a simple discovery request: https://gist.githubusercontent.com/vincenthsu/6eb1f9b2c07d123c526b/raw/0fc216bcba8ec55b5db92c1e6f9c922b836ccc49/onvif_request.py Using postman to send an HTTP request, I send the discovery message, and the stated error message that I receive is coming directly from the camera.

POST to IP:8899:/onvif/device_service

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope">
    <s:Header>
        <Security s:mustUnderstand="0" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
            <UsernameToken>
                <Username>admin</Username>
                <Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">xHRDiSKpw45Az6r1A6eTs1nYqds=</Password>
                <Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">qGpR6gxNRF1Qk174PkggkAhLIvE=</Nonce>
                <Created xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2022-03-16T07:29:29.000Z</Created>
            </UsernameToken>
        </Security>
    </s:Header>
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <GetCapabilities xmlns="http://www.onvif.org/ver10/device/wsdl">
            <Category>All</Category>
        </GetCapabilities>
    </s:Body>
</s:Envelope>

receive:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:chan="http://schemas.microsoft.com/ws/2005/02/duplex" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:c14n="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:wsc="http://schemas.xmlsoap.org/ws/2005/02/sc" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:xmime="http://tempuri.org/xmime.xsd" xmlns:xop="http://www.w3.org/2004/08/xop/include" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl" xmlns:tmd="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:trc="http://www.onvif.org/ver10/recording/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl">
    <SOAP-ENV:Body>
        <SOAP-ENV:Fault>
            <SOAP-ENV:Code>
                <SOAP-ENV:Value>SOAP-ENV:MustUnderstand</SOAP-ENV:Value>
            </SOAP-ENV:Code>
            <SOAP-ENV:Reason>
                <SOAP-ENV:Text xml:lang="en">The data in element 'Security' must be understood but cannot be handled</SOAP-ENV:Text>
            </SOAP-ENV:Reason>
        </SOAP-ENV:Fault>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Taking into account the error message, by changing the "mustUnderstand" field to 0 in the request, I can receive this reply:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:chan="http://schemas.microsoft.com/ws/2005/02/duplex" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:c14n="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:wsc="http://schemas.xmlsoap.org/ws/2005/02/sc" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:xmime="http://tempuri.org/xmime.xsd" xmlns:xop="http://www.w3.org/2004/08/xop/include" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl" xmlns:tmd="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:trc="http://www.onvif.org/ver10/recording/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl">
    <SOAP-ENV:Header></SOAP-ENV:Header>
    <SOAP-ENV:Body>
        <tds:GetCapabilitiesResponse>
            <tds:Capabilities>
                <tt:Device>
                    <tt:XAddr>http://192.168.1.14:8899/onvif/device_service</tt:XAddr>
                    <tt:Network>
                        <tt:IPFilter>false</tt:IPFilter>
                        <tt:ZeroConfiguration>false</tt:ZeroConfiguration>
                        <tt:IPVersion6>false</tt:IPVersion6>
                        <tt:DynDNS>false</tt:DynDNS>
                    </tt:Network>
                    <tt:System>
                        <tt:DiscoveryResolve>true</tt:DiscoveryResolve>
                        <tt:DiscoveryBye>true</tt:DiscoveryBye>
                        <tt:RemoteDiscovery>false</tt:RemoteDiscovery>
                        <tt:SystemBackup>false</tt:SystemBackup>
                        <tt:SystemLogging>false</tt:SystemLogging>
                        <tt:FirmwareUpgrade>false</tt:FirmwareUpgrade>
                        <tt:SupportedVersions>
                            <tt:Major>2</tt:Major>
                            <tt:Minor>2</tt:Minor>
                        </tt:SupportedVersions>
                    </tt:System>
                    <tt:IO>
                        <tt:InputConnectors>1</tt:InputConnectors>
                        <tt:RelayOutputs>1</tt:RelayOutputs>
                    </tt:IO>
                    <tt:Security>
                        <tt:TLS1.1>false</tt:TLS1.1>
                        <tt:TLS1.2>false</tt:TLS1.2>
                        <tt:OnboardKeyGeneration>false</tt:OnboardKeyGeneration>
                        <tt:AccessPolicyConfig>false</tt:AccessPolicyConfig>
                        <tt:X.509Token>false</tt:X.509Token>
                        <tt:SAMLToken>false</tt:SAMLToken>
                        <tt:KerberosToken>false</tt:KerberosToken>
                        <tt:RELToken>false</tt:RELToken>
                    </tt:Security>
                </tt:Device>
                <tt:Media>
                    <tt:XAddr>http://192.168.1.14:8899/onvif/Media</tt:XAddr>
                    <tt:StreamingCapabilities>
                        <tt:RTPMulticast>true</tt:RTPMulticast>
                        <tt:RTP_TCP>true</tt:RTP_TCP>
                        <tt:RTP_RTSP_TCP>true</tt:RTP_RTSP_TCP>
                    </tt:StreamingCapabilities>
                </tt:Media>
                <tt:PTZ>
                    <tt:XAddr>http://192.168.1.14:8899/onvif/PTZ</tt:XAddr>
                </tt:PTZ>
                <tt:Extension>
                    <tt:DeviceIO>
                        <tt:XAddr>http://192.168.1.14:8899/onvif/DeviceIO</tt:XAddr>
                        <tt:VideoSources>1</tt:VideoSources>
                        <tt:VideoOutputs>0</tt:VideoOutputs>
                        <tt:AudioSources>0</tt:AudioSources>
                        <tt:AudioOutputs>0</tt:AudioOutputs>
                        <tt:RelayOutputs>1</tt:RelayOutputs>
                    </tt:DeviceIO>
                </tt:Extension>
            </tds:Capabilities>
        </tds:GetCapabilitiesResponse>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

so it seems that the camera is 'working' but just not very good. Any workaround? I don't know anything about this but why not just always set it to zero? It kind of looks to me like the standard allows for an initial unauthenticated GetCapabilites anyways: https://www.onvif.org/wp-content/uploads/2016/12/ONVIF_Feature_Discovery_Specification_16.07.pdf page 20:

ONVIF Client invokes GetCapabilitiesRequest message without any authentication to retrieve the capabilities and check GetCapabilities command support by DUT.

ispysoftware commented 2 years ago

yeah i can change that to 0. I'm a bit concerned it might break other onvif devices that want it to be 1 though - it's working fine on my own onvif cameras. Are you able to send me an IP/ login for it for testing? I'd rather not deploy this until i know it's working.

timmeh87 commented 2 years ago

It is understandable that such a change is not to be taken lightly given that it affects the discovery of all ONVIF cameras and It has been in service for years now. I think the standard compromise is to add it to the GUI with the default option being the one that worked before. That way nobody has a bad day and everyone can access the workaround if they want it. I can work on isolating a camera if you wish. right now they are mounted to the ceiling and looking at my family, I would prefer to test a new feature myself. I do have the ability to build things myself so if its possible to do a branch and build it here I can also do that, not sure what toolchain you are using. Is this... F-sharp?

On Wed, Mar 16, 2022 at 4:30 AM sean tearney @.***> wrote:

yeah i can change that to 0. I'm a bit concerned it might break other onvif devices that want it to be 1 though - it's working fine on my own onvif cameras. Are you able to send me an IP/ login for it for testing? I'd rather not deploy this until i know it's working.

— Reply to this email directly, view it on GitHub https://github.com/ispysoftware/iSpy/issues/177#issuecomment-1068862341, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTCBFVH6CXRINEQU7BJPJTVAGL2PANCNFSM5QZ4DL5A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

timmeh87 commented 2 years ago

Oh also I should probably mention at this point what camera I have in case it could help someone https://www.aliexpress.com/item/1005002891688995.html?spm=a2g0o.order_list.0.0.21ef1802iDvDF8

On Wed, Mar 16, 2022 at 11:42 AM Tim B @.***> wrote:

It is understandable that such a change is not to be taken lightly given that it affects the discovery of all ONVIF cameras and It has been in service for years now. I think the standard compromise is to add it to the GUI with the default option being the one that worked before. That way nobody has a bad day and everyone can access the workaround if they want it. I can work on isolating a camera if you wish. right now they are mounted to the ceiling and looking at my family, I would prefer to test a new feature myself. I do have the ability to build things myself so if its possible to do a branch and build it here I can also do that, not sure what toolchain you are using. Is this... F-sharp?

On Wed, Mar 16, 2022 at 4:30 AM sean tearney @.***> wrote:

yeah i can change that to 0. I'm a bit concerned it might break other onvif devices that want it to be 1 though - it's working fine on my own onvif cameras. Are you able to send me an IP/ login for it for testing? I'd rather not deploy this until i know it's working.

— Reply to this email directly, view it on GitHub https://github.com/ispysoftware/iSpy/issues/177#issuecomment-1068862341, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTCBFVH6CXRINEQU7BJPJTVAGL2PANCNFSM5QZ4DL5A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

ispysoftware commented 2 years ago

yeah i just dont know if it's going to fix your issue until i can test it. I don't want to go to the hassle of building out config and ui and pushing a release on the off chance it's going to fix access to your camera. Agent isn't open source so I can't give you access to the code - if you want me to add this I'll need to be able to test it. Just point it at the ceiling or something for 20 minutes.

timmeh87 commented 2 years ago

Ok we can do that. I can prepare a camera so you have more than 20 minutes. I am pretty sure that I am behind CG-nat though so I think Im going to have to do some testing on my end and come up with a way to actually do that. I think I might need to use ipv6 and run some kind of proxy inside my network. Ill do some testing and make sure i can remotely access it from outside my own network and let you know when its ready

On Wed, Mar 16, 2022 at 9:47 PM sean tearney @.***> wrote:

yeah i just dont know if it's going to fix your issue until i can test it. I don't want to go to the hassle of building out config and ui and pushing a release on the off chance it's going to fix access to your camera. Agent isn't open source so I can't give you access to the code - if you want me to add this I'll need to be able to test it. Just point it at the ceiling or something for 20 minutes.

— Reply to this email directly, view it on GitHub https://github.com/ispysoftware/iSpy/issues/177#issuecomment-1069822755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTCBFWR6YI5JHRXOFPPFTDVAKFK3ANCNFSM5QZ4DL5A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

timmeh87 commented 2 years ago

It turns out that whatever username and password I thought I had set on this camera with the v380 android app was not necessary ... all streams and features are accessible with no username and password at all. I HAD been using the streams with a username and password in the URL but it seems to just ignore it, it can be literally anything and it still works. Im not sure if the camera is crappy or I just dont understand how to use it. (maybe both)

Now, I never tried this before but If I perform the camera search with the username and password field blank, it finds the ONVIF camera url! I dont actually need my cameras to have passwords on my home network so Ill just leave it like this. I can PTZ them which Is all I wanted

Which means I have a working solution, no changes needed. Thanks for taking the time to look at this with me

ispysoftware commented 2 years ago

the latest upload set the mustUnderstand to 0 - haven't had any reports yet of it breaking things.

timmeh87 commented 2 years ago

Oh wow, awesome. Thanks for doing the update. I was trying to get my camera ports forwarded for the last week and it became a whole 'thing' leading to a crash course in ipv6 and the replacement of my main router.

I originally reported this working with version 3.8.6.0, so the "no username or password at all" method worked already with the old software! I have signed up for a month of paid and downloaded the software update, now I am on 3.8.9.0 and my cameras work with any fake password as well. literally anything. Still not sure if I just failed to set a password on these or what... I suppose that would be consistent with what I have been seeing. That would be a pretty big "d'oh"

one last thought about this, if a given camera actually cared about the password, and someone had mistyped their password, would it initially scan okay with the wrong password due to this change (no auth required to do initial scan) and then later fail, due to a wrong password? Given what I know now, I will probably just connect my cameras with no login (or else learn how to properly configure the password!) and if this change had to be reverted down the line I would still be okay

ispysoftware commented 2 years ago

it'd find the camera but the connection to the video stream should fail - unless the camera authentication is broken, which is quite possible :)

timmeh87 commented 2 years ago

I had one more thought, it could be an automatic process whereby when this exact error is received, discovery is re-attempted without mandatory authentication. Then zero user intervention and it "just works" for everyone

Relevant XKCD https://xkcd.com/1172/

On Wed, Mar 16, 2022 at 11:45 AM Tim B @.***> wrote:

Oh also I should probably mention at this point what camera I have in case it could help someone

https://www.aliexpress.com/item/1005002891688995.html?spm=a2g0o.order_list.0.0.21ef1802iDvDF8

On Wed, Mar 16, 2022 at 11:42 AM Tim B @.***> wrote:

It is understandable that such a change is not to be taken lightly given that it affects the discovery of all ONVIF cameras and It has been in service for years now. I think the standard compromise is to add it to the GUI with the default option being the one that worked before. That way nobody has a bad day and everyone can access the workaround if they want it. I can work on isolating a camera if you wish. right now they are mounted to the ceiling and looking at my family, I would prefer to test a new feature myself. I do have the ability to build things myself so if its possible to do a branch and build it here I can also do that, not sure what toolchain you are using. Is this... F-sharp?

On Wed, Mar 16, 2022 at 4:30 AM sean tearney @.***> wrote:

yeah i can change that to 0. I'm a bit concerned it might break other onvif devices that want it to be 1 though - it's working fine on my own onvif cameras. Are you able to send me an IP/ login for it for testing? I'd rather not deploy this until i know it's working.

— Reply to this email directly, view it on GitHub https://github.com/ispysoftware/iSpy/issues/177#issuecomment-1068862341, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTCBFVH6CXRINEQU7BJPJTVAGL2PANCNFSM5QZ4DL5A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>