Depth mode ULTRA slows performance even on stream-type without depth

[alexander-jiang]

Comparing the same gstreamer pipeline with depth-mode=0 (NONE) explicitly set (despite stream-type=0 aka left view only), the performance is significantly better with depth-mode=0 set (suggesting that the depth calculation is being performed despite no depth view being retrieved). And since the default depth mode is ULTRA, the performance impact is significant.

gst-launch-1.0 -e  zedsrc stream-type=0 resolution=1 framerate=15 depth-mode=0 enable-positional-tracking=FALSE ! queue ! videoconvert ! omxh265enc control-rate=2 bitrate=10000000 ! h265parse ! qtmux ! filesink location=no_depth_pipeline.mp4 

gst-launch-1.0 -e  zedsrc stream-type=0 resolution=1 framerate=15 enable-positional-tracking=FALSE ! queue ! videoconvert ! omxh265enc control-rate=2 bitrate=10000000 ! h265parse ! qtmux ! filesink location=depth_pipeline.mp4 

Testing on my Jetson Nano, the output video with depth-mode=0 hits ~14 fps (according to ffprobe -i), but the output video without depth-mode explicitly set to NONE is only 7-9fps.

[alexander-jiang]

This was a change I introduced in my PR, I'll submit another to make the default depth-mode = NONE

[alexander-jiang]

One note: since the default value for the enable-positional-tracking property of zedsrc is true, setting the default depth mode to NONE will cause an error unless enable-positional-tracking=false is explicitly set.

Perhaps a more appropriate default depth mode is PERFORMANCE? Or should the default for enable-positional-tracking be false instead? Or should the user be made aware of the computational cost of the depth mode, even when the stream-type is set such that there is no depth data retrieved from the camera?

@Myzhar Please advise

[Myzhar]

@alexander-jiang I'm studying these combinations of parameter values to better understand how to set all of them. What I think is that if depth-mode is NONE, all the other parameters that depend on depth must be forced to the correct value. i.e. positional tracking cannot be activated, so a "warning" must be issued if its value is found as TRUE

[Tetsujinfr]

Initially I was a bit annoyed by this too but then I realised it made it clear I needed to get a bit more involved with the gstreamer camera parameters, which I think is a good thing. I appreciate those requires a bit more investment but I think it is good to have depthmode to ultra as a default, so that one pays attention to those params. Just sharing a different user view on this, and not sure there is any issue here really. Maybe adding a bit more documentation or examples on the readme of this repo would made the trick really.

[alexander-jiang]

I agree with @Tetsujinfr, adding some more documentation or examples would probably be enough, I just wanted to point this out as I was also not very clear about how the different zedsrc properties were related (and how leaving depth mode enabled would impact performance, even in a stream-type without depth data).

[Myzhar]

This ticket is currently under working in the devel branch. Depth mode has now a default value of NONE.

To disable the depth elaboration it is also necessary to disable the positional tracking and the object detection, so the values of the relative activation options have been set to false as default. The zedsrc element now starts without performing any depth elaboration with the default settings.

DEPTH_MODE is forced to ULTRA (if not differently specified by setting depth-mode) in all the following cases:

• stream-type is 3 or 4
• enable-positional-tracking is true
• od-enabled is true NOTE: od-enabled forces enable-positional-tracking to true if it's not.

NOTE: a warning is issued when one of the parameters is forced to a value different from the default value. To log all the warnings it is necessary to set the DEBUG level to a value major than 2, for example: GST_DEBUG=zedsrc:4 gst-launch-1.0 zedsrc od-enabled=true ! autovideoconvert ! autovideosink

[Myzhar]

The new features have been merged in the devel branch. Issue closed

[Tetsujinfr]

Thanks for the dev, always good to see Stereolabs listening to the user base. Keep it going, would be so great to have a dev forum on the website or something like that so that the discussion on features/requests is easier.

[obraun-sl]

Hi, We have started one here : https://community.stereolabs.com/

There is no direct link on the website for the moment, but that's just a matter of days...

[Tetsujinfr]

That is really cool, thanks for sharing. Got so many ideas for your product I would like to discuss. Excellent! Merci