Crashes, when click "scan"

[icho40]

Hi, I have tried to compile the Manager under MX-Linux. When I click on "Scan" in the app, the app crashes.

INFO 23-12-31 18:21:36.462135 139753307056832 onvif_app.c:138: Starting ONVIF Devices Network Discovery...
ProbMatches__create...
ProbMatch__create...
segmentation fault

[icho40]

It did, imou-cam is working now.

[Quedale]

That's great news!

What about the yi cam? That one should be working too after the fix applied to OnvifSoapLib.

The log file you attached on March 5th shows that you didn't have the latest changes.

[icho40]

The Yi-cams still don't work. odm.log

[Quedale]

Okay, the good news is that the fallback mechanism did work. You can see that it did retrieve the RTSP url and the snapshot url.

The bad news is that I forgot to also implement this fallback mechanism to the snapshot retrieval and the RtspPlayer. At least I know the solution works.

Let me get on it.

[Quedale]

Your yi cam should work now.

Try this

git pull
./autogen.sh --enable-latest --prefix=$(pwd)/dist
make clean
make -j$(nproc)
./onvifmgr

Note that this is a fallback mechanism after it initially fails to connect. Some cameras may use dynamically generated URI, so I can't safely cache this information to re-use.

For that reason this camera may respond a bit slower than others. The best I would be able to offer to mitigate this, is to add an application settings to control the timeout value. Reducing timeout value would ultimately increase your camera's responsiveness in the application.

I still recommend that you look into why your camera is returning the wrong ip address within its soap responses.

Looking forward to hear back!

[icho40]

willi@icho:~/onvif/OnvifDeviceManager >echo $PROBE_DEBUG; echo $ONVIF_DEBUG 1 1

all manually added: yi-1 shows cam-icon with Exclamation mark, but no picture. yi-2 shows cam-icon with Exclamation mark, but no picture. yi-outdoor works, but also shows cam-icon with Exclamation mark imou works (but only manually added, scan crashes it. yi1-debug.txt yi2-debug.txt yi-outdoor-debug.txt

[icho40]

I still recommend that you look into why your camera is returning the wrong ip address within its soap responses.

How can I do that, I'm not a software developer. I had already suggested asking the programmer of the alternative firmware for the Yi-Cams.

[Quedale]

Okay, let's fix the video stream not working, which should be relatively easy.

It seems both your cameras are using an unexpected audio codec and the gstreamer plugin is missing. If we can figure out what it is, I can add it. This line was extracted from your log files:

gstrtspplayer.c:823: Unhandled element msg name : missing-plugin)

run the application with GST_DEBUG=GST_EVENT*:4. This will make gstreamer output its logs.

GST_DEBUG=GST_EVENT*:4 ./onvifmgr

The resulting logs should contain something like "creating caps event audio/x...." Here's what one of my camera returns.

creating caps event audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)8000, channels=(int)1

If you can provide that, I will add the missing plugin to the gstreamer build.

Thanks again for helping so much.

[icho40]

I have now run the application with GST_DEBUG=GST_EVENT*:4. I have attached the log files. I noticed that the expression alaw appears there. I remembered that you can set disabled, pcm, alaw, ulaw and acc for RTSP audio in the camera's web interface. After I switched all cameras to pcm, it worked for all of them. Only with the Yi cameras I have no preview image, although I can find the link to it in the web interface of the cameras. yi1-debug.txt yi2-debug.txt yi-outdoor-debug.txt

after changing RTSP audio to pcm: debug.txt

[Quedale]

Alright, I just added the gstalaw plugin, which should resolve your original audio codec issue. I also added some logging to display supported audio codec as well now. Here's an example:

INFO  24-03-14 11:18:45.077330 123975775140544 onvif-mgr.c:126: **** Audio decoders *******
INFO  24-03-14 11:18:45.077337 123975775140544 onvif-mgr.c:49: * audio/x-mulaw decoders ***
INFO  24-03-14 11:18:45.077434 123975775140544 onvif-mgr.c:58: *    mulawdec[256]
INFO  24-03-14 11:18:45.077445 123975775140544 onvif-mgr.c:49: * audio/x-alaw decoders ***
INFO  24-03-14 11:18:45.077542 123975775140544 onvif-mgr.c:58: *    alawdec[256]
INFO  24-03-14 11:18:45.077554 123975775140544 onvif-mgr.c:49: * audio/mpeg decoders ***
INFO  24-03-14 11:18:45.077651 123975775140544 onvif-mgr.c:58: *    fdkaacdec[64]
INFO  24-03-14 11:18:45.077664 123975775140544 onvif-mgr.c:130: ****************************

Alaw codec should work now and I can easily add more codecs if needed.

Now, regarding the broken thumbnail! I can only see this:

onvif_media_service.c:273: [http://192.168.178.52/onvif/media_service] Snapshot URI : http://192.168.178.52/cgi-bin/snapshot.sh?res=high
omgr_device_row.c:459: Error writing png to GtkPixbufLoader : [null]

I just added a new debugging flag SNAP_DEBUG, which will help us troubleshoot this specific issue. This will make it dump HTTP request/response for the snapshot retrieval.

Try this:

git pull
./autogen.sh --enable-latest --prefix=$(pwd)/dist
make clean
make -j$(nproc)
SNAP_DEBUG=1 ./onvifmgr

If you could share a (uncompressed) snapshot retrieved from the URL http://192.168.178.52/cgi-bin/snapshot.sh?res=high, I may be able to reproduce the problem locally.

[icho40]

I have tested with the version from yesterday tested all 4 possible audio variants: RTSP audio: "Disabled" works "pcm" works "alaw" does not work (see alaw-audio-debug.txt) "ulaw" works "aac" works

alaw-audio-debug.txt

Then I did this:

git pull
./autogen.sh --enable-latest --prefix=$(pwd)/dist
make clean
make -j$(nproc)
SNAP_DEBUG=1 ./onvifmgr

and received /usr/bin/ld: ./src/queue/onvifmgr-event_queue.o: in function EventQueue__remove_thread: /home/willi/onvif/OnvifDeviceManager/./src/queue/event_queue.c:114: undefined reference to CListTS__destroy_record collect2: Error: ld returned 1 exit status make: *** [Makefile:1076: onvifmgr] error 1

I also downloaded a snapshot with wget.

[Quedale]

and received /usr/bin/ld: ./src/queue/onvifmgr-event_queue.o: in function EventQueue__remove_thread: /home/willi/onvif/OnvifDeviceManager/./src/queue/event_queue.c:114: undefined reference to CListTS__destroy_record collect2: Error: ld returned 1 exit status make: *** [Makefile:1076: onvifmgr] error 1

I completely forgot to commit the changes to CUtils. (That's my bad, sorry!) Because of that, the changes were not applied.

Try this again:

git pull
./autogen.sh --enable-latest --prefix=$(pwd)/dist
make clean
make -j$(nproc)
SNAP_DEBUG=1 ./onvifmgr

I tried the picture provided and it worked on my setup. I'm hoping that SNAP_DEBUG will provide additional details.

Thanks so much!

[icho40]

Here you go: imou-debug.txt yi1-debug.txt yi2-debug.txt yi-outdoor-debug.txt

[Quedale]

Yi Cameras all show a successfull HTTP 200 response, although doesn't seem to return any data.

gtkbinaryimage.c:116: GtkBinaryImage__new - size 0
gtkbinaryimage.c:122: Unable to create GtkBinaryImage with NULL data.

Unfortunately this doesn't tell me much. (Obviously something went wrong during the HTTP call)

imou-debug.txt

onvif_media_service.c:256: [http://192.168.178.53:80/onvifsnapshot/media_service/snapshot?channel=1&subtype=0] Failed to retrieve snapshot: soap_smd_final() failed [No context][30]

This error is probably the most relevant one indicating that I probably miss-handle the gsoap pointer. That prompted me to review the code thoroughly and I did bring some modifications.

More specifically, the client will now close the socket after the server returned a 401, before sending the authentication request.

As usual, to test it:

git pull
./autogen.sh --enable-latest --prefix=$(pwd)/dist
make clean
make -j$(nproc)
SNAP_DEBUG=1 ./onvifmgr

Best of luck!

[icho40]

Imou-Ranger works fine added manually: yes with password: yes with thumbnail: yes Imou-Ranger-debug.txt

Yi-01 works added manually: yes with password: no with thumbnail: no yi-01-debug.txt

Yi-02 works added manually: yes with password: yes with thumbnail: no yi-02-debug.txt

Yi-outdoor works added manually: yes with password: no with thumbnail: no yi-outdoor-debug.txt

[Quedale]

I may have an idea on the cause, but I don't think the fault is within this project.

I actually looked at the source code of your yi-hack firmware and noticed something that got patched/improved on a fork.

Here's what I found: I was able to find the snapshot webservice code found here: snapshot.sh. Inside that script, you can see that it invokes imggrabber -m $MODEL $RES $WATERMARK.

Imggrabber code is available here imggrabber.c. From that code, I was able to see that it plain and simply read the content of the camera sensor. Most video devices have exclusive access and I would suspect the video sensor of yi cameras are too.

Basically, my software starts the stream and fetch the snapshot simultaneously after manually adding it. I think imggrabber fails the snapshot because the camera sensor is in use by the live stream.

The reason I know about this, is because rpos suffers from the same issue on the raspberry pi.

What happens if you try to open http://1234:1234@192.168.178.51/cgi-bin/snapshot.sh?res=high on your browser, WHILE the video stream is playing on my software?

It seems Yi-Hack-MStar worked around this issue by leveraging some kind rotating snapshot cache. See getlastrecordedsnapshot.sh, which relies on a cached version instead of accessing the video sensor directly.

Interestingly, roleoroleo owns both repository. I'm not sure why there's such a discrepancy between them.

That being said, I shouldn't need to retrieve the snapshot while the stream is playing. I could easily create a snapshot from the live stream instead, saving a lot of resources in the process

Let me see what I can do to enhance this.

[Quedale]

I've got some good news!

I got my hands on a Imou Ranger 2C and it suffers the same discovery issue as yours. I managed to pin the root cause to some xml namespace that Imou doesnt handle properly.

Unfortunately, Im not sure yet how to gracefully resolve it, but I did get the Imou camera to discover on a dirty test case.

I also got a Yi camera that I didnt setup yet. I should be able to test it soon.

[icho40]

This is really good news. Thank you for staying on top of this.

[Quedale]

I just committed the fix for the Imou camera discovery.

In short, I simply defined a hardcoded namespace definition for the discovery. (see here) I couldn't figure out how to prevent gSoap from padding the message with unnecessary namespaces.

The Imou camera didn't like the following 5 namespaces required for regular soap calls.

        { "saml1", "urn:oasis:names:tc:SAML:1.0:assertion", NULL, NULL },
        { "saml2", "urn:oasis:names:tc:SAML:2.0:assertion", NULL, NULL },
        { "wsu", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd", NULL, NULL },
        { "xenc", "http://www.w3.org/2001/04/xmlenc#", NULL, NULL },
        { "wsc", "http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512", "http://schemas.xmlsoap.org/ws/2005/02/sc", NULL },

I will start working on supporting YiHack soon.

You will need to run autogen.sh once again after pulling the changes.

[icho40]

willi@icho:~/onvif/OnvifDeviceManager >git pull
Already current.

willi@icho:~/onvif/OnvifDeviceManager >./autogen.sh --enable-latest --prefix=$(pwd)/dist
configure.ac:3: error: Autoconf version 2.70 or higher is required
configure.ac:3: the top level
autom4te: /usr/bin/m4 failed with exit status: 63
aclocal: error: /usr/bin/autom4te failed with exit status: 63
autoreconf: aclocal failed with exit status: 63
*****************************
*** Autogen failed libx11 ***
*****************************

willi@icho:~/onvif/OnvifDeviceManager >autoconf -V
autoconf (GNU Autoconf) 2.69

[Quedale]

I did introduce new dependencies when I added OpenGL rendering. It seems your distro doesn't have the appropriate autoconf version installed to compile this dependency.

Can you try to install you repo's "libx11-xcb-dev" package?

autogen.sh should skip it if it's already installed.

[Quedale]

I also figured out why the Yi snapshots aren't working, but no fix yet.

Here are the headers for the Yi snapshot:

* Connected to 192.168.1.18 (192.168.1.18) port 80 (#0)
> GET /cgi-bin/snapshot.sh?res=high HTTP/1.1
> Host: 192.168.1.18
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-type: image/jpeg
* no chunk, no close, no size. Assume close to signal end
< 

I think the key problem is the missing Content-Length or Transfer-Encoding: chunked.

I will see what can be done to properly handle this (like modern browsers do), but I think it may be worth reporting as a yihack issue.

EDIT: Removed wrong logs and updated suspicions with latest finding.

[icho40]

I did introduce new dependencies when I added OpenGL rendering. It seems your distro doesn't have the appropriate autoconf version installed to compile this dependency.

Can you try to install you repo's "libx11-xcb-dev" package?

autogen.sh should skip it if it's already installed.

Sorry, didn't work.

autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal -I m4
configure.ac:3: error: Autoconf version 2.70 or higher is required
configure.ac:3: the top level
autom4te: /usr/bin/m4 failed with exit status: 63
aclocal: error: /usr/bin/autom4te failed with exit status: 63
autoreconf: aclocal failed with exit status: 63
*****************************
*** Autogen failed libx11 ***
*****************************

willi@icho:sudo apt install libx11-xcb-dev
Package lists are read... Done
Dependency tree is created... Done
Status information is read in... Done
libx11-xcb-dev is already the latest version (2:1.7.2-2~mx21ahs+1).

I found this page, But I'm not sure how to compile it. http://ftp.gnu.org/gnu/autoconf/

[Quedale]

Thanks for providing the logs. I lowered the minimum libx11 version from 1.8.6 to 1.7.2 to support your distro without fuss.

Pull the latest changes and autogen.sh should accept your libx11 version.

[Quedale]

I just added a workaround to handle snapshot service that doesn't behave properly like yihack.

Let me know how that goes!

EDIT: Turns out, the yihack snapshot behaviour is within the HTTP spec, although not recommended.

RFC 7230 Section 3.3.3

   7.  Otherwise, this is a response message without a declared message
       body length, so the message body length is determined by the
       number of octets received prior to the server closing the
       connection.

   Since there is no way to distinguish a successfully completed,
   close-delimited message from a partially received message interrupted
   by network failure, a server SHOULD generate encoding or
   length-delimited messages whenever possible.  The close-delimiting
   feature exists primarily for backwards compatibility with HTTP/1.0.

Therefor, this isn't a bug in yihack, but its a bug in gsoap HTTP_DA plugin. I think I found a relatively graceful way to handle it.

Cheers!

[icho40]

I just committed the fix for the Imou camera discovery.

Works flawlessly

Therefor, this isn't a bug in yihack, but its a bug in gsoap HTTP_DA plugin. I think I found a relatively graceful way to handle it.

That's good to hear

[Quedale]

Works flawlessly

That's great news!

Is there any remaining issue? Are we good to close it?