gstreamer-1.0-1.19.3-x86_64 on macOS

[stephanschulz]

Thanks for making this addon.

I successfully managed to compile and run the example on macOS 10.15.7 with OF 0.11.2 using gstreamer-1.0-1.10.2-x86_64.pkg and gstreamer-1.0-devel-1.10.2-x86_64.pkg

I did notice the latency was a bit much and thought to install gstreamer-1.0-devel-1.20.1-universal by editing the install.sh. This resulted in gst.h not being found. I thought maybe universal was not a good pick. (Not sure what it means :) ) So I tried gstreamer-1.0-devel-1.19.3-x86_64. The app compiles but when trying to start I now get this error message

dyld: Library not loaded: /Users/matt/Projects/cerbero-1.20/build/dist/darwin_universal/x86_64/lib/GStreamer
  Referenced from: /Applications/of_v0.11.2_osx_release/apps/network-camera/videoGrabberExample/bin/videoGrabberExample.app/Contents/MacOS/videoGrabberExample
  Reason: image not found

Usually that means something was not included in to the .app file.

Would anyone know how to fix this? Thanks a bunch.

[arturoc]

I guess the config for the latest release might have changed. In any case I don't think the latency would be solved by upgrading. You can try running a gstreamer pipeline from the console to see if the latency there is the same with the new version before deciding if it's worth trying to solve the issue.

There's also certain parameters that can be passed to gstreamer when creating the pipeline that can reduce the latency in exchange for worse scrubbing or other problems

[stephanschulz]

Thanks for the reply. Is the config a specific file and if so where would I find it and reset it?

I am working with an Axis security camera that has a build in mic. I am hoping to access video + audio with as little latency as possible. I tired using VLC with rtsp://198.168.0.101/axis-media/media.amp and got a similar amount of latency. Using ofxIpVideoGrabber and just requesting the image stream via http://198.168.0.101/axis-cgi/mjpg/video.cgi?resolution=800x600 works without latency. So maybe the latency is more due to the .amp format.

Besten Gruß.

[arturoc]

With config i mean the inlcudes and libraries that need to be linked with this addon. The config file to change that for each addon is addon_confg.mk

A while ago I used an axis camera for a project using gstreamer and did this addon that must set everything up and internally uses http requests to control the parameters of the camera from a gui: https://github.com/arturoc/ofxAxisGrabber/

It's 9 years old so it surely won't work out of the box but should be relatively straightforward to fix

[stephanschulz]

Yes I saw the addon but have not yet tried it. Do you remember if you used audio with it? It seems only video works very fast but adding audio causes delays.

[arturoc]

No, it uses a custom gstreamer pipeline which seems to only use video, adding audio should be relatively straightforward

[stephanschulz]

There's also certain parameters that can be passed to gstreamer when creating the pipeline that can reduce the latency

I wonder if you can share an example on how to manipulate these parameters? I saw this but it's unclear how to tap in to it from ofApp.

Also tried this processing sketch. With latency = 0 the results very pretty acceptable.

[stephanschulz]

I think I managed to find a solution and documented it here: https://forum.openframeworks.cc/t/reducing-video-latency/5931/24?u=stephanschulz

[arturoc]

Great, you can also do that from outside the core source while setting the pipeline like the ofxAxisGrabber addon does here https://github.com/arturoc/ofxAxisGrabber/blob/master/src/ofxAxisGrabber.cpp#L108

[stephanschulz]

I saw that but I had a feeling that maybe the gStreamer in the OF code base seemed more up to date. And since I am pretty new to the subject I was easier to work with than creating new life in to ofxAxisGrabber. Thank you again for all your advice.

[stephanschulz]

I am learning that having added the g_object_set(G_OBJECT(gstPipeline), "latency", 0, (void*)NULL); line effects the audio a lot that I am getting from the axis camera. I am noted it here too: https://forum.openframeworks.cc/t/audio-analysis-on-video/4605/7

Even if I set latency to 2000 I get bad audio. Only if I remove the above line does the audio sound good again. I guess setting latency forces a specific timing but it rather needs a dynamic setting. But how?