Crashes, when click "scan"

[icho40]

Hi, I have tried to compile the Manager under MX-Linux. When I click on "Scan" in the app, the app crashes.

INFO 23-12-31 18:21:36.462135 139753307056832 onvif_app.c:138: Starting ONVIF Devices Network Discovery...
ProbMatches__create...
ProbMatch__create...
segmentation fault

[Quedale]

Good day and thanks for this feedback!

I didn't get a chance to work on this project for a while, but I'll try to allocate some times to improve it over the coming weeks. People showing interest is always a great motivation!

I think I already found the root cause of this problem and two unrelated minor memory leak using valgrind. A regression was introduced within the last big changes I committed inside the new logging framework.

Here are my finding if you are interested: The issue is in the file clogger.c inside the method c_log. The thread lock is applied after parsing the level string and datetime when it should be at the beginning.

In short, while a line is being printed using the level and datetime pointer, those pointers may be rewritten by another thread trying to print a different line.

I will finalize the changes soon and let you know once it's ready for testing.

Thanks!

[Quedale]

I've spent some time with valgrind hunting down leaks and fixed a considerable number. Most were harmless given that they are only cleaned up when closing the application.

I did also solve a few invalid memory access and race conditions, which should resolve your issue.

After pulling the latest changes, make sure to run autogen.sh once again in order to pull the latest changes from dependencies. The changes were applied to OnvifDeviceManager, OnvifDiscoveryLib, OnvifSoapLib and CUtils.

Let me know how that goes!

[icho40]

Hello, thanks for the quick reply. I have deleted the "OnvifDeviceManager" directory and re-downloaded the repository with git clone https://github.com/Quedale/OnvifDeviceManager.git cd OnvifDeviceManager and then recompiled according to your instructions. Unfortunately, the result is the same: DEBUG 24-01-05 00:50:08.719508 140077754038976 app_dialog.c:125: hide INFO 24-01-05 00:50:09.823687 140077754038976 onvif_app.c:147: Starting ONVIF Devices Network Discovery... ProbMatches__create... ProbMatch__create... segmentation fault

[Quedale]

I'm pretty sure I figured it out now.

I didn't encounter this issue because I'm using plain IP(s) in my environment. After testing with a camera using hostname, I encountered the same crash.

I just committed the patch under the project OnvifSoapLib.

You can either nuke the entire thing and start fresh like you did before , or simply run the autogen.sh script before recompiling.

I'm pretty confident that fixed it.

[icho40]

Unfortunately, it still does not work with the same error:

DEBUG 24-01-05 10:50:21.912731 140265679468288 queue_thread.c:25: Started...
DEBUG 24-01-05 10:50:21.921129 140266042854080 onvif_details.c:108: clear_details
TRACE 24-01-05 10:50:22.057386 140266042854080 gui_utils.c:12: gui_update_widget_image_priv
TRACE 24-01-05 10:50:22.057411 140266042854080 gui_utils.c:29: gui_update_widget_image_priv done
INFO  24-01-05 10:50:24.250403 140266042854080 onvif_app.c:147: Starting ONVIF Devices Network Discovery...
ProbMatches__create...
ProbMatch__create...
egmentation fault

Is there a way to increase the debug level?

[Quedale]

You can go ahead and give it another try. I made another big round of needed changes which may fix your issue.

I did increase logging on the discovery mechanism.

I also added the ability to set "PROBE_DEBUG" environment variable at runtime in order to dump discovery soap messages. e.g.: PROBE_DEBUG=1 ./onvifmgr This dump will allow me to simulate the same response you are getting from your camera.

Don't forget to rerun autogen.sh after pulling the changes.

Thanks for the feedback!

[Quedale]

You can hold it a little bit. I setup a MX Linux VM and managed to reproduce the problem.

Let me dig further into it.

[Quedale]

Alright, I got it running smoothly on a MX Linux VM with the last few commits.

Looking forward to hear back!

[icho40]

Unfortunately, it crashes still. Is there any more information i can give you?

[Quedale]

Please run the application with the PROBE_DEBUG environment variable set. This will print the entire soap message exchanged between the client and the camera(s).

Share these messages and I should be able to reproduce it locally.

[icho40]

Please run the application with the PROBE_DEBUG environment variable set.

How do i do this?

[Quedale]

Instead of running the application like this ./onvifmgr you run it like this PROBE_DEBUG=1 ./onvifmgr

Alternatively, you can do it in two seperate line:

export PROBE_DEBUG=1
./onvifmgr

[icho40]

ok, i did so.

willi@icho:~/OnvifDeviceManager >PROBE_DEBUG=1 ./onvifmgr
DEBUG 24-01-08 00:37:17.031782 140331359688192 onvifinitstaticplugins.c:222: Initializing Gstreamer plugins...
DEBUG 24-01-08 00:37:17.031941 140331359688192 onvifinitstaticplugins.c:227: Loading static plugins...
DEBUG 24-01-08 00:37:17.041802 140331359688192 onvifinitstaticplugins.c:433: Gstreamer plugins initialized...
INFO  24-01-08 00:37:17.041826 140331359688192 onvif-mgr.c:40: Using Gstreamer Version : 1.22.8.0
TRACE 24-01-08 00:37:17.041835 140331359688192 app_dialog.c:65: init
TRACE 24-01-08 00:37:17.051563 140331359688192 credentials_input.c:28: create
TRACE 24-01-08 00:37:17.051578 140331359688192 app_dialog.c:65: init
TRACE 24-01-08 00:37:17.052738 140331359688192 msg_dialog.c:60: create
TRACE 24-01-08 00:37:17.052747 140331359688192 app_dialog.c:65: init
TRACE 24-01-08 00:37:17.053234 140331359688192 event_queue.c:83: create...
INFO  24-01-08 00:37:17.057169 140331359688192 app_settings.c:188: Reading Settings file onvifmgr_settings.ini
WARN  24-01-08 00:37:17.057194 140331359688192 app_settings.c:245: No config file found. Using default configs. 1
INFO  24-01-08 00:37:17.060696 140331359688192 backchannel.c:111: Creating backchannel using source element pulsesrc
DEBUG 24-01-08 00:37:17.060970 140331264440064 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.060995 140331247585024 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.061037 140331239192320 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.061090 140331222406912 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.061287 140331230799616 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.061327 140331009963776 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.061413 140331001571072 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.061446 140330993178368 queue_thread.c:25: Started...
DEBUG 24-01-08 00:37:17.067074 140331359688192 onvif_details.c:108: clear_details
TRACE 24-01-08 00:37:17.088804 140331359688192 gui_utils.c:12: gui_update_widget_image_priv
TRACE 24-01-08 00:37:17.088841 140331359688192 gui_utils.c:29: gui_update_widget_image_priv done
INFO  24-01-08 00:37:19.206343 140331359688192 onvif_app.c:167: Starting ONVIF Devices Network Discovery...
INFO  24-01-08 00:37:19.207045 140330970539776 onvif_discovery.c:120: PROBE_DEBUG variable set. '1'
TRACE 24-01-08 00:37:19.207263 140330970539776 onvif_discovery.c:165: Sending Device probe...
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:tdn="http://www.onvif.org/ver10/network/wsdl"><SOAP-ENV:Header><wsa:MessageID>urn:uuid:9efa8b17-1787-4e12-ab8b-4567327b23c6</wsa:MessageID><wsa:ReplyTo SOAP-ENV:mustUnderstand="true"><wsa:Address>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:Address></wsa:ReplyTo><wsa:To SOAP-ENV:mustUnderstand="true">urn:schemas-xmlsoap-org:ws:2005:04:discovery</wsa:To><wsa:Action SOAP-ENV:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/Probe</wsa:Action></SOAP-ENV:Header><SOAP-ENV:Body><wsdd:Probe xmlns:_0="http://www.onvif.org/ver10/device/wsdl"><wsdd:Types>_0:Device</wsdd:Types></wsdd:Probe></SOAP-ENV:Body></SOAP-ENV:Envelope>
TRACE 24-01-08 00:37:19.207425 140330970539776 onvif_discovery.c:177: Sending NVT probe...
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:tdn="http://www.onvif.org/ver10/network/wsdl"><SOAP-ENV:Header><wsa:MessageID>urn:uuid:9efa8b17-1787-4e12-ab8b-4567327b23c6</wsa:MessageID><wsa:ReplyTo SOAP-ENV:mustUnderstand="true"><wsa:Address>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:Address></wsa:ReplyTo><wsa:To SOAP-ENV:mustUnderstand="true">urn:schemas-xmlsoap-org:ws:2005:04:discovery</wsa:To><wsa:Action SOAP-ENV:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/Probe</wsa:Action></SOAP-ENV:Header><SOAP-ENV:Body><wsdd:Probe><wsdd:Types>tdn:NetworkVideoTransmitter</wsdd:Types></wsdd:Probe></SOAP-ENV:Body></SOAP-ENV:Envelope>
TRACE 24-01-08 00:37:19.207504 140330970539776 onvif_discovery.c:188: Sending NVD probe...
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:tdn="http://www.onvif.org/ver10/network/wsdl"><SOAP-ENV:Header><wsa:MessageID>urn:uuid:9efa8b17-1787-4e12-ab8b-4567327b23c6</wsa:MessageID><wsa:ReplyTo SOAP-ENV:mustUnderstand="true"><wsa:Address>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:Address></wsa:ReplyTo><wsa:To SOAP-ENV:mustUnderstand="true">urn:schemas-xmlsoap-org:ws:2005:04:discovery</wsa:To><wsa:Action SOAP-ENV:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/Probe</wsa:Action></SOAP-ENV:Header><SOAP-ENV:Body><wsdd:Probe><wsdd:Types>tdn:NetworkVideoDisplay</wsdd:Types></wsdd:Probe></SOAP-ENV:Body></SOAP-ENV:Envelope>
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/10" xmlns:pub="http://schemas.microsoft.com/windows/pub/2005/07" xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:wsdp="http://schemas.xmlsoap.org/ws/2006/02/devprof" xmlns:wsx="http://schemas.xmlsoap.org/ws/2004/09/mex"><soap:Header><wsa:To>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:To><wsa:Action>http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:Action><wsa:MessageID>urn:uuid:63f32f2d-d407-813b-eeb0-1a25de966f3a</wsa:MessageID><wsa:RelatesTo>urn:uuid:9efa8b17-1787-4e12-ab8b-4567327b23c6</wsa:RelatesTo><wsd:AppSequence InstanceId="1" MessageNumber="4489" SequenceId="urn:uuid:1d7aa09e-3a8e-b3f6-9ff4-6f807e7d40d1" /></soap:Header><soap:Body><wsd:ProbeMatches><wsd:ProbeMatch><wsa:EndpointReference><wsa:Address>urn:uuid:da4a86d1-4362-e82c-a162-d06d1cb8c478</wsa:Address></wsa:EndpointReference><wsd:Types>wsdp:Device pub:Computer</wsd:Types><wsd:MetadataVersion>1</wsd:MetadataVersion></wsd:ProbeMatch></wsd:ProbeMatches></soap:Body></soap:Envelope>TRACE 24-01-08 00:37:19.213581 140330970539776 onvif_discovery.c:62: ProbeMatches found...
TRACE 24-01-08 00:37:19.213597 140330970539776 probmatch.c:15: ProbMatches__create...
TRACE 24-01-08 00:37:19.213603 140330970539776 probmatch.c:84: ProbMatch__create...
Speicherzugriffsfehler

[Quedale]

Thank you so much for this.

The key take away I get from this is that this isn't an ONVIF device responding. Or not a compliant one at minimum.

Here's what you get formatted, which contains no XAddrs field, which is mandatory for ONVIF.

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope
    xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/10"
    xmlns:pub="http://schemas.microsoft.com/windows/pub/2005/07"
    xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
    xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"
    xmlns:wsd="http://schemas.xmlsoap.org/ws/2005/04/discovery"
    xmlns:wsdp="http://schemas.xmlsoap.org/ws/2006/02/devprof"
    xmlns:wsx="http://schemas.xmlsoap.org/ws/2004/09/mex">
    <soap:Header>
        <wsa:To>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:To>
        <wsa:Action>http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:Action>
        <wsa:MessageID>urn:uuid:63f32f2d-d407-813b-eeb0-1a25de966f3a</wsa:MessageID>
        <wsa:RelatesTo>urn:uuid:9efa8b17-1787-4e12-ab8b-4567327b23c6</wsa:RelatesTo>
        <wsd:AppSequence InstanceId="1" MessageNumber="4489" SequenceId="urn:uuid:1d7aa09e-3a8e-b3f6-9ff4-6f807e7d40d1" />
    </soap:Header>
    <soap:Body>
        <wsd:ProbeMatches>
            <wsd:ProbeMatch>
                <wsa:EndpointReference>
                    <wsa:Address>urn:uuid:da4a86d1-4362-e82c-a162-d06d1cb8c478</wsa:Address>
                </wsa:EndpointReference>
                <wsd:Types>wsdp:Device pub:Computer</wsd:Types>
                <wsd:MetadataVersion>1</wsd:MetadataVersion>
            </wsd:ProbeMatch>
        </wsd:ProbeMatches>
    </soap:Body>
</soap:Envelope>

For reference, here's a typical ONVIF NVT device response:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
    xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope"
    xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"
    xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery"
    xmlns:dn="http://www.onvif.org/ver10/network/wsdl">
    <SOAP-ENV:Header>
        <wsa:MessageID>uuid:cd450610-adb6-11ee-8e8a-97afecbb290b</wsa:MessageID>
        <wsa:RelatesTo>urn:uuid:b9850019-1787-4e12-ab8b-4567327b23c6</wsa:RelatesTo>
        <wsa:To SOAP-ENV:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:To>
        <wsa:Action SOAP-ENV:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:Action>
        <d:AppSequence SOAP-ENV:mustUnderstand="true" MessageNumber="68" InstanceId="1460972484"/>
    </SOAP-ENV:Header>
    <SOAP-ENV:Body>
        <d:ProbeMatches>
            <d:ProbeMatch>
                <wsa:EndpointReference>
                    <wsa:Address>urn:uuid:83d95616-eb60-501a-ba8e-cfc4e17b8a5c</wsa:Address>
                </wsa:EndpointReference>
                <d:Types>dn:NetworkVideoTransmitter</d:Types>
                <d:Scopes>
                    onvif://www.onvif.org/Profile/T
                    onvif://www.onvif.org/type/video_encoder
                    onvif://www.onvif.org/type/ptz
                    onvif://www.onvif.org/hardware/P52%20Protected
                    onvif://www.onvif.org/name/Lenovo%20P52%20Protected
                    onvif://www.onvif.org/location/
                  </d:Scopes>
                <d:XAddrs>http://192.168.1.184:8082/onvif/device_service</d:XAddrs>
                <d:MetadataVersion>1</d:MetadataVersion>
            </d:ProbeMatch>
        </d:ProbeMatches>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Note how your response doesn't contain any onvif.org schema definition, onvif scopes or xaddrs defined.

I'm not sure what this device is or why it operates on the same port as ONVIF discovery (3702), but this is actually a very good finding. This software should be resilient to unexpected responses from the network.

I'll get a fix for it soon.

Thanks for this valuable feedback!

[icho40]

That is strange, because with https://flathub.org/apps/net.meijn.onvifviewer it looks like this:

[Quedale]

I expect those devices to show up on this software as well.

I just didn't anticipate non-ONVIF response, but I should have. I should simply ignore invalid matches, which is most likely what ONVIFViewer does.

[Quedale]

Okay. It's finally fixed. I managed to reproduce the exact same crash and properly resolve it.

Give it a shot now! :thumbsup:

[icho40]

Just to make sure that I have not made a mistake: I have deleted the directory "OnfivDeviceManager" and "/usr/local/bin/onvifmgr" Downloaded the repository with "git clone https://github.com/Quedal/OnvifDeviceManager.git" Then executed "./autogen.sh", "make" and "sudo make install". Then: "export PROBE_DEBUG=1", followed by "onvifmgr" This is new:

6:49.598397 139843358062336 onvif_discovery.c:62: ProbeMatches found... TRACE 24-01-08 07:56:49.598407 139843358062336 probmatch.c:15: ProbMatchescreate... WARN 24-01-08 07:56:49.598412 139843358062336 onvif_discovery.c:80: Invalid response. Ingnoring...(Use 'PROBE_DEBUG' environment variable to troubleshoot) WARN 24-01-08 07:56:49.603163 139843810215680 device.c:94: Failed to get hostname ... INFO 24-01-08 07:56:49.603186 139843810215680 device.c:55: NetBIOS Lookup ... 192.168.178.103 INFO 24-01-08 07:56:49.608111 139843810215680 device.c:74: Retrieved hostname : MC563-A1547D.fritz.box TRACE 24-01-08 07:56:49.608132 139843810215680 device.c:125: _priv_Devicelookup_hostname - done TRACE 24-01-08 07:56:49.608140 139843810215680 event_queue.c:176: notify dispatched event... Speicherzugriffsfehler

and

:ProbeMatch></wsd:ProbeMatches></soap:Body></soap:Envelope>WARN 24-01-08 08:13:38.048518 139655190132480 device.c:94: Failed to get hostname ... INFO 24-01-08 08:13:38.048553 139655190132480 device.c:55: NetBIOS Lookup ... 192.168.178.103 TRACE 24-01-08 08:13:38.048570 139655215310592 onvif_discovery.c:62: ProbeMatches found... TRACE 24-01-08 08:13:38.048579 139655215310592 probmatch.c:15: ProbMatchescreate... WARN 24-01-08 08:13:38.048584 139655215310592 onvif_discovery.c:80: Invalid response. Ingnoring...(Use 'PROBE_DEBUG' environment variable to troubleshoot) WARN 24-01-08 08:13:38.051970 139655206917888 device.c:94: Failed to get hostname ... INFO 24-01-08 08:13:38.051996 139655206917888 device.c:55: NetBIOS Lookup ... 192.168.178.112 INFO 24-01-08 08:13:38.052007 139655190132480 device.c:74: Retrieved hostname : MC563-A1547D.fritz.box TRACE 24-01-08 08:13:38.052022 139655190132480 device.c:125: _priv_Devicelookup_hostname - done TRACE 24-01-08 08:13:38.052029 139655190132480 event_queue.c:176: notify dispatched event... INFO 24-01-08 08:13:38.056507 139655206917888 device.c:74: Retrieved hostname : imouranger.fritz.box TRACE 24-01-08 08:13:38.056547 139655206917888 device.c:125: _priv_Devicelookup_hostname - done TRACE 24-01-08 08:13:38.056597 139655206917888 event_queue.c:176: notify dispatched event... DEBUG 24-01-08 08:13:38.194596 139655198525184 onvif_device.c:69: Created Media soap [http://192.168.178.112/onvif/media_service] TRACE 24-01-08 08:13:38.194639 139655198525184 onvif_media_service.c:273: OnvifMediaServiceget_profile_token INFO 24-01-08 08:13:38.194653 139655198525184 onvif_media_service.c:48: OnvifMediaService__get_profile_token : Initializing profiles Speicherzugriffsfehler

[icho40]

maybe this helps: While executing autogen.sh, i redirected stdout to autogen.txt. Then i got this on stderr:

willi@icho:~/OnvifDeviceManager >./autogen.sh > autogen.txt Package gstreamer-1.0 was not found in the pkg-config search path. Perhaps you should add the directory containing gstreamer-1.0.pc' to the PKG_CONFIG_PATH environment variable No package 'gstreamer-1.0' found Package gstreamer-plugins-base-1.0 was not found in the pkg-config search path. Perhaps you should add the directory containinggstreamer-plugins-base-1.0.pc' to the PKG_CONFIG_PATH environment variable No package 'gstreamer-plugins-base-1.0' found Package gstreamer-plugins-good-1.0 was not found in the pkg-config search path. Perhaps you should add the directory containing gstreamer-plugins-good-1.0.pc' to the PKG_CONFIG_PATH environment variable No package 'gstreamer-plugins-good-1.0' found Package gstreamer-plugins-bad-1.0 was not found in the pkg-config search path. Perhaps you should add the directory containinggstreamer-plugins-bad-1.0.pc' to the PKG_CONFIG_PATH environment variable No package 'gstreamer-plugins-bad-1.0' found

[Quedale]

Those autogen errors aren't really errors, since it should fallback and build the library from source.

The good news is that it did go past the discovery stage and attempted to query a network camera. I was looking for an official ONVIF device demo stream online and managed to pull this one:

URL: http://61.216.97.157:16887/onvif/device_service User: demo Password: demo

Funny enough, this camera doesn't even seem properly compliant, since it doesn't even work properly on the official ONVIF Device Manager client. Regardless, this is an excellent use-case to improve the resiliency.

I will let you know once the new set of changes are available.

Thank you so much for your patience!

[Quedale]

I've made another big round of changes in order to properly support the demo camera. Hopefully this will resolve your issues.

Looking forward to hear back

[icho40]

With the demo cam it works with channel 1 and 2, with channel 4 I get no picture, but this error message:

(onvifmgr:256884): GStreamer-WARNING **: 13:20:42.574: ../subprojects/gstreamer/gst/gstpad.c:5389:store_sticky_event: Sticky event misordering, got 'segment' before 'caps'

Unfortunately, I don't know the device URL or the port for my cameras. I have tried to guess it but unfortunately without success:

URL: http://192.168.178.131:80/onvif/device_service i get:

INFO 24-01-11 13:32:05.860588 140307485374208 onvif_device.c:178: protocol -- http INFO 24-01-11 13:32:05.860591 140307485374208 onvif_device.c:179: ip : 192.168.178.131 INFO 24-01-11 13:32:05.860595 140307485374208 onvif_device.c:180: hostname : (null) INFO 24-01-11 13:32:05.860599 140307485374208 onvif_device.c:181: port -- 80 INFO 24-01-11 13:32:05.860602 140307485374208 onvif_device.c:182: endpoint : onvif/devic_service ERROR 24-01-11 13:32:05.860608 140307485374208 onvif_app.c:536: Invalid URL provided

URL: http://192.168.178.131:80/onvif/media_service i get:

INFO 24-01-11 13:35:34.972407 139898462615296 onvif_device.c:177: Created Device: INFO 24-01-11 13:35:34.972409 139898462615296 onvif_device.c:178: protocol -- http INFO 24-01-11 13:35:34.972412 139898462615296 onvif_device.c:179: ip : 192.168.178.131 INFO 24-01-11 13:35:34.972414 139898462615296 onvif_device.c:180: hostname : (null) INFO 24-01-11 13:35:34.972416 139898462615296 onvif_device.c:181: port -- 80 INFO 24-01-11 13:35:34.972418 139898462615296 onvif_device.c:182: endpoint : onvif/media_service ERROR 24-01-11 13:35:34.972422 139898462615296 onvifapp.c:536: Invalid URL provided DEBUG 24-01-11 13:35:34.972425 139898462615296 onvif

Is there a way, i can provide you more information?

[icho40]

I've noticed something else that's strange.

I write the stdout to yi-errors.txt with

onvifmgr > yi-errors.txt

and open it with

tail -f yi-errors.txt

When I try to add this URL http://192.168.178.130:80/onvif/device_service I get messages and then the output stops. If I then click on "add" again, the output continues at exactly the same point and then stops again. This goes on and on.

See in the file in the lines 13, 51, 89, 127, 167, 204, 244, 282, 319, 359, 398, 420, 460, 499, 536, 575, 613, 652 yi-errors.txt

[Quedale]

First, this warning is generated by the gstreamer library. Such warning can happen with unstable camera feed for instance. I'll dig further to possibly handle this more gracefully, but this isn't a camera on your LAN so this is more likely to happen.

(onvifmgr:256884): GStreamer-WARNING **: 13:20:42.574: ../subprojects/gstreamer/gst/gstpad.c:5389:store_sticky_event:rtpjpegdepay0:src Sticky event misordering, got 'segment' before 'caps'

Here, you had a typo on this attempt. You typed "devic_service" instead of "device_service".

URL: http://192.168.178.131:80/onvif/device_service i get:

INFO 24-01-11 13:32:05.860588 140307485374208 onvif_device.c:178: protocol -- http INFO 24-01-11 13:32:05.860591 140307485374208 onvif_device.c:179: ip : 192.168.178.131 INFO 24-01-11 13:32:05.860595 140307485374208 onvif_device.c:180: hostname : (null) INFO 24-01-11 13:32:05.860599 140307485374208 onvif_device.c:181: port -- 80 INFO 24-01-11 13:32:05.860602 140307485374208 onvif_device.c:182: endpoint : onvif/devic_service <--TYPO ERROR 24-01-11 13:32:05.860608 140307485374208 onvif_app.c:536: Invalid URL provided

All official ONVIF devices need an entry point "http(s)://host:port/onvif/device_service". Although you don't need it when using the discovery feature. (I don't see you trying it from the log file)

Manually adding camera isn't really a finished feature yet. Since there's no loading indicator showing that something is happening, or preventing the user from clicking the button multiple times. (That's my next thing) It could also be friendlier by asking appropriate fields like IP, port, user, password instead of a full URL...

In the last commit, I added a new environment variable flag "ONVIF_DEBUG" in order to generate soap message dump. (Very similar to "PROBE_DEBUG").
IMPORTANT: Make sure to cleanup sensitive data like username and hashed password from the output

Feel free to enable both environment variable to get a full debug output.

ONVIF_DEBUG=1 \
PROBE_DEBUG=1 \
./onvifmgr

or

export ONVIF_DEBUG=1
export PROBE_DEBUG=1
./onvifmgr

I find this very confusing. According to the log file provided, it successfully queried "http://192.168.178.130:80/onvif/device_service" and than proceeded to invoke "http://192.168.178.13:80/onvif/media_service" which returned a connection error.
(Note that the media_service URL is provided by the device_service, I didn't come up with it)

I'm suspecting that I may need to add some wait period between the calls. The camera may not be able to keep up.

[icho40]

Do you think it makes sense to contact the author of the Yi hack about this issue? https://github.com/roleoroleo/yi-hack-Allwinner-v2#features

[Quedale]

Not yet, it would be too early to assume a faulty camera. Note that a slow camera isnt a faulty camera. (And it may not even be the problem)

If you can get the ONVIF soap messages like you did before for the discorvery, that would he really helpful.

[icho40]

Do you mean these: yi-errors.txt

[Quedale]

Just to give a status update.

I indentified a key issue related to the authentication mechanism. Simply put, I dont sync the clock between the client and the camera.

A camera receiving authentication with a time with a too great offset compared to its internal clock should be refused. This is actually part of the ONVIF developper guide.

Fortunately, I just received an extremely cheap camera to work with affected by this issue.

I will let you know once it's available.

[Quedale]

I've made another big round of changes including crash fix.

Let me know how that goes!

Thanks for your patience.

[icho40]

After ./autogen.sh make sudo make install

i got this message:

~/OnvifDeviceManager >sudo make install CC src/app/onvifmgr-device.o ./src/app/device.c: In function ‘_priv_Devicelookup_hostname_netbios’: ./src/app/device.c:54:21: warning: implicit declaration of function ‘OnvifDeviceget_ip’; did you mean ‘OnvifDeviceget_port’? [-Wimplicit-function-declaration] 54 | char * dev_ip = OnvifDeviceget_ip(device->onvif_device); | ^~~~~~~ | OnvifDeviceget_port ./src/app/device.c:54:21: warning: initialization of ‘char ’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] ./src/app/device.c: In function ‘_priv_Device__lookup_hostname_dns’: ./src/app/device.c:84:21: warning: initialization of ‘char ’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] 84 | char * dev_ip = OnvifDeviceget_ip(device->onvif_device); | ^~~~~~~ ./src/app/device.c: In function ‘_priv_Devicelookup_hostname’: ./src/app/device.c:115:21: warning: initialization of ‘char ’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] 115 | char dev_ip = OnvifDeviceget_ip(device->onvif_device); | ^~~~~~~ ./src/app/device.c:117:9: warning: implicit declaration of function ‘OnvifDevice__lookup_ip’ [-Wimplicit-function-declaration] 117 | OnvifDevicelookup_ip(device->onvif_device); | ^~~~~~ ./src/app/device.c: In function ‘_priv_Deviceload_thumbnail’: ./src/app/device.c:157:25: error: ‘ONVIF_NOT_AUTHORIZED’ undeclared (first use in this function); did you mean ‘ONVIF_ERROR_NOT_AUTHORIZED’? 157 | } else if(oerror == ONVIF_NOT_AUTHORIZED){ | ^~~~~~~~ | ONVIF_ERROR_NOT_AUTHORIZED ./src/app/device.c:157:25: note: each undeclared identifier is reported only once for each function it appears in ./src/app/device.c: In function ‘gui_Devicedisplay_profiles’: ./src/app/device.c:264:66: error: ‘ONVIF_NOT_AUTHORIZED’ undeclared (first use in this function); did you mean ‘ONVIF_ERROR_NOT_AUTHORIZED’? 264 | if(OnvifDeviceget_last_error(evt->device->onvif_device) == ONVIF_NOT_AUTHORIZED){ | ^~~~~~~~ | ONVIF_ERROR_NOT_AUTHORIZED ./src/app/device.c: In function ‘_priv_Deviceload_profiles’: ./src/app/device.c:302:61: error: ‘ONVIF_NOT_AUTHORIZED’ undeclared (first use in this function); did you mean ‘ONVIF_ERROR_NOT_AUTHORIZED’? 302 | if(OnvifDeviceget_last_error(device->onvif_device) == ONVIF_NOT_AUTHORIZED){ | ^~~~~~~~ | ONVIF_ERROR_NOT_AUTHORIZED ./src/app/device.c: In function ‘Devicecreate_row’: ./src/app/device.c:381:21: warning: initialization of ‘char ’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] 381 | char dev_ip = OnvifDeviceget_ip(device->onvif_device); | ^~~~~~~ make: *** [Makefile:1221: src/app/onvifmgr-device.o] Fehler 1

[Quedale]

I think you may have forgot to execute "git pull" before running autogen.sh.

For instance src/app/device.c:54:21, this line is now commented out.

[icho40]

Sorry, my bad. I'm not a developer, I use git quite rarely. I'll try again in a moment.

[icho40]

debug.txt Now only my Imou cam is found, my Yi-cams are not. But I can't log in.

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>s:Sender</s:Value>ter:NotAuthorized</s:Value></s:Subcode></s:Code>Sender not Authorized. Invalid username or password! You still have 5 attempt(s).</s:Text></s:Reason></s:Fault></s:Body></s:Envelope>WARN 24-01-31 16:32:55.661472 140312009762560 onvif_base_service.c:189: Warning : NOT AUTHORIZED HTTP Error code [12]

It also finds my printer.

PrintDeviceType wscn:ScanDeviceType</wsd:Types>http://192.168.178.34/

[Quedale]

Thanks for testing it. I'm glad it got past crashing.

Invalid credenials

If I understand correctly, the Imou camera is found, but credentials are not accepted? I must ask the most obvious question. Are you 100% sure the credentials are correct? (Does it work on the official ODM?) Also, you may have sent the wrong log file, because I don't see this error in it.

I did make some modification related to the authentication mechanism. I'll have another look at it.

Yi cameras

In regards of the Yi-cams, are you sure they are ONVIF capable and discoverable? (Do they show up on the official ODM?) I did notice that my new very cheap camera stops responding to discovery probe after a short while. I have to restart it to restore the discovery feature. Although the camera can still be manually added.

Note that Yi technology is an IoT platform unrelated to ONVIF. A camera can be compatible with both Yi IoT and ONVIF. (Same goes for Imou and Tuya)

Printer Showing Up

Filtering out the printer from discovery should be an easy fix. I just need to filter probe matches based on scopes or stop probing for simple devices. The funny thing, this is plain WS-Discovery and I mimic the probing behaviour from the official ODM. Does the printer show up on it? (If not I can assume ODM filters out this response)

The log file

I see only 2 probe response [192.168.178.133 & 192.168.178.34], which I assume is the Imou cam and the printer. It doesn't look like any response from the Yi cam was received. Although, it does seem that you tried to manually add a camera without the "http://" part of the URL:

24-01-31 16:56:43.814476 139913215321856 onvif_device.c:144: Created Device:
24-01-31 16:56:43.814478 139913215321856 onvif_device.c:145:    protocol -- (null)
24-01-31 16:56:43.814479 139913215321856 onvif_device.c:146:    host : 192.168.178.130
24-01-31 16:56:43.814481 139913215321856 onvif_device.c:147:    port -- 80
24-01-31 16:56:43.814483 139913215321856 onvif_device.c:148:    endpoint : onvif/device_service

Improving the dialog to avoid these mistakes is on my TODO list.

[icho40]

Are you 100% sure the credentials are correct?

Yes, i made an factory reset and used the given credentials.

(Does it work on the official ODM?

on net.meijn.onvifviewer yes

Also, you may have sent the wrong log file, because I don't see this error in it.

I don't really know, which file you need.

In regards of the Yi-cams, are you sure they are ONVIF capable and discoverable?

yes, i am, because i flasht the Yi-Hack https://github.com/roleoroleo/yi-hack-Allwinner-v2

(Do they show up on the official ODM?)

on net.meijn.onvifviewer yes

The funny thing, this is plain WS-Discovery and I mimic the probing behaviour from the official ODM. Does the printer show up on it?

No

[Quedale]

So I've been having issues with my WIFI AP where it will allow UDP broadcast only for a short period after the camera connected to the wifi. My wifi AP seems to "forget" its wireless clients and stops forwarding UDP broadcasts, although the wifi still works perfectly fine. (This is just an assumption) I'm using a popular cheap wifi router as a dedicated access point without DHCP server.

I was able to solve my issue by simply turning on Multicast Stream on my Wifi AP :

Can you make sure that this setting is enable on your wifi router/AP ?

onvifviewer has the luxury of remembering devices, so only one successful discovery is sufficient. What is interesting from onvifviewer's behavior is that the device will still show up on the discovery panel if the device was previously setup. Can you try to remove all the cameras from onvifviewer and discovering them again see if that works?

Note that saving discovered and manual added cameras is a feature I will be looking at soon.

[icho40]

Can you make sure that this setting is enable on your wifi router/AP ?

I use a Fritz.Box 7530 AX. I can't find anything in the settings to activate or deactivate multicast. But since it is a current router and Internet TV is also running, I assume that it is already activated.

Can you try to remove all the cameras from onvifviewer and discovering them again see if that works?

Only my Imou camera was recognized, but I was able to add the Yi cameras manually with their IP address

[Quedale]

I use a Fritz.Box 7530 AX. I can't find anything in the settings to activate or deactivate multicast. But since it is a current router and Internet TV is also running, I assume that it is already activated.

In my case, the multicast settings was only applied to the wifi AP feature of the router. The physical ports (switch feature) was not affected.

Only my Imou camera was recognized, but I was able to add the Yi cameras manually with their IP address

I expected this. I'll be working on camera configuration persistence next.

1. Manually added camera will be automatically saved.
2. Discovered cameras will have the ability to be saved for future sessions.

To sum up, the only issue you are experiencing right now is that it won't authenticate against your Imou camera?

Does the yi camera work when you add it manually? Use the full url "http://192.168.178.130/onvif/device_service". (including "http://") I'll work on making that simpler as well.

[icho40]

To sum up, the only issue you are experiencing right now is that it won't authenticate against your Imou camera?

There seems to be a misunderstanding. This worked only in the Flatpak net.meijn.onvifviewer

With your onvifviewer only the Imou camera was recognized, but refused the credentials.

Does the yi camera work when you add it manually?

on net.meijn.onvifviewer: yes on your viewer: no

debug.txt

[Quedale]

Okay sorry, yea we got a bit confused. I did bring additional fixes and improved logging once more. (One fix in particular related to authentication)

Would you mind if we close this issue considering that the original one was resolved along the way?

You can create new separate ones for the issues remaining, that would be appreciated. Using new and separate issues would shorten and simplify our threads.

Thanks!

[icho40]

ok, thanks for your patience

[Quedale]

No, thank YOU for this valuable feedback!

Im looking forward for additional feedback.

[Quedale]

I probably found the problem with the credentials being refused.

Related to issue #19 .

You have the same log in the file you provided.

HTTP/1.1 401 Unauthorized
Connection: close
Content-Length: 199
Content-Type: application/soap+xml; charset=utf-8
CSeq: 0
WWW-Authenticate: Digest realm="Login to FB161248B5877600",qop="auth",nonce="83CD9007325F46603738F3DCAA7060426420B51F169CD5BE151FC1A2362D7BE", opaque="", stale="false"

Support is coming.

[Quedale]

I just added support for HTTP authentication challenge. It should simultaneously support Basic, Digest and NTLM authentication scheme.

Simply pull the latest changes and run autogen.sh once again to pull and build the latest changes from OnvifSoapLib as well.

I was unfortunately only able to test Basic authentication without given the effort required to setup an appropriate test environment.

Let me know how that goes.

Thank for your help!

[icho40]

Hi, I have completely deleted the OnvifDeviceManager folder and recloned the repository, both the DeviceManager and the OnvifSoapLib.

Unfortunately, my cameras are still not found, via search. Adding them manually does not work either. What information do you need?

[Quedale]

You dont need to anything related to dependencies.

To update the repo AND dependency you can do this:

git pull
./autogen.sh --enable-latest
make

The autogen.sh take care of the dependencies.

That being said, the fix isnt related to the discovery, but the authenication.

You should now be able to manually add the camera tho.

Can you provide a new log file?

[icho40]

Unfortunately, OnvifManager now denies access to both the Imou and Yi-cams.

imou-error.log yi-error.log

[icho40]

I have now done the following: OnvifSoapLib:

git pull cd build cmake -DCMAKE_INSTALL_PREFIX="$(pwd)/dist" .. make -j$(nproc)

the "make" resulted in errors result: make-error.log make-error.log

OnvifDeviceManager: git pull` ./autogen.sh --enable-latest make

willi@icho: >echo $PROBE_DEBUG 1 willi@icho: >echo $ONVIF_DEBUG 1

willi@icho:./onvifmgr 2>&1> odm.log I tried: yi-cam without pw yi-cam with pw yi-outdoor without pw imou-cam without pw: Unauthorized imou-cam with pw: crashes result: odm.log: odm.log imou-error.log: imou-error.log

[Quedale]

You are confusing two project.

From the perspective of OnvifDeviceManager, autogen.sh already takes care of the dependencies on its own. Because you actually went inside OnvifSoapLib and performed the update manually, the autogen.sh didn't pick up the changes.

You CAN update OnvifSoapLib manually, but you missed two key steps that the autogen.sh script does for you:

git pull
cd build
../bootstrap.sh
cmake -DCMAKE_INSTALL_PREFIX="$(pwd)/dist" ..
make -j$(nproc)
make install

For the OnvifSoapLib project, bootstrap.sh takes care of the dependencies. In this case, CUtils also needed to be updated. The line make install is also required to setup files under "./subprojects/OnvifSoapLib/build/dist", which is where OnvifDeviceManager looks for the library.

What you can do at this point to recover :

rm -rf ./subprojects/OnvifSoapLib/build/dist/*
./autogen.sh --enable-latest
make

Basically, you simply remove the previously compiled library and force autogen.sh to rebuild it.

Thanks for not giving up!

[Quedale]

I just managed to reproduce and fix a crash when a snapshot was loaded.

That might resolve the issue related to your imou camera. Issue #22 experienced the exact same thing.