SDK 3.8 Update + QoL Changes

[ryanppeters]

Updates the plugin to add the new features from the the ZED SDK v3.8 release, namely:

1. Adds Camera::setRegionOfInterest function as the zedsrc properties roi, roi-x, roi-y, roi-w, and roi-h
2. Adds ObjectDetectionParameters::allow_reduced_precision_inference as the zedsrc property od-allow-reduced-precision-inference
3. Adds PositionalTrackingParameter::set_floor_as_origin as the zedsrc property set-gravity-as-origin
4. Adds PositionalTrackingParameters::depth_min_range as the zedsrc property pos-depth-min-range
5. Adds ObjectDetectionParaeters::prediction_timeout_sas thezedsrcpropertyod-prediction-timeout-s`

Adds a frame_id field to GstZedSrcMeta in order to track the meta/buffer throughout the GStreamer pipeline (when working with source code).

Fixes a bug in the initial world transform that input roll, pitch, yaw in the same order regardless of the coordinate system being used

Notes:

1. I copied the descriptions for the new zedsrc parameters from the API reference but some of them may still not be the best
2. Our implementation of frame_id could probably be improved and welcome any redesign of it, but we find it necessary for our code to have the ability to track buffers throughout the pipeline. We work with NVidia Deepstream via CPP and found that some of Nvidia's GStreamer elements "mess-up" the information in the base GstBuffer which prevents us from using that to track the Buffer throughout the pipeline
3. We may have missed some edge cases if any of the new parameters are mutually exclusive with other ones

[Myzhar]

Hi @ryanppeters thank you for the pull request. I quickly analyzed the modifications and everything seems well integrated. I must ask you for more information about the frame_id field of the metadata and how you initialize it. Regarding instead the ROI, I saw that you used a basic rectangle to define it. This can be good for a first implementation, but in the future, we could change it into a more generic polygon like we already did in the ROS2 wrapper:

Setting: https://github.com/stereolabs/zed-ros2-wrapper/blob/master/zed_wrapper/config/common.yaml#L22-L25 ROI creation: https://github.com/stereolabs/zed-ros2-wrapper/blob/master/zed_components/src/tools/src/sl_tools.cpp#L420-L478

[ryanppeters]

I must ask you for more information about the frame_id field of the metadata and how you initialize it.

frame_id is set from the return value of GST_BUFFER_OFFSET(buf) which is "a media specific offset for the buffer data. For video frames, this is the frame number of this buffer..." (source). We see this holds true for ZedSrc at GstZedSrc Lines2509-210. The GST_BUFFER_OFFSET(buf) call occurs at GstZedSrc.cpp Lines2513-2514 with offset being the value for the frame_id to be set in the gst_buffer_add_zed_src_meta(...). We choose to keep calculating the frame_id in GstZedSrc rather than GstZedSrcMeta for clarity relating to (1) passing all GstZedSrcMeta fields at once and (2) getting the offset just after it was updated inside ZedSrc by GstZedSrc Lines2509-210.

Unfortunately, some NVidia Elements (we believe to be either NvStreamMux or NvInfer) are modifying the offset member in GstBuffer such that we cannot reliably use GST_BUFFER_OFFSET(...) to track GstBuffers through the pipeline, hence the need to add this in GstZedSrcMeta instead.

[Myzhar]

@ryanppeters fixed and merged into master.

Thank you for the great PR :+1:

[ryanppeters]

Thanks for merging and glad we can help!

I did some additional testing with Deepstream and determined that it is NvStreamMux that changes the GstBuffer offset (sets it to GST_BUFFER_OFFSET_NONE). This makes sense since it is batching together 1+ buffers into a new GstBuffer. Although we only use one source (ZED2i) in our use case, NvStreamMux still batches it causing the offset to change.

I will add that after doing more research and testing, I discovered that the NvDsFrameMeta from the output NvDsBatchMeta of NvStreamMux has a field frame_num that could be potentially used to track the buffers without needing the extra frame_id in GstZedSrcMeta. The GstZedSrcMeta.frame_id and NvDsFrameMeta.frame_num solutions both work well for single source pipelines with Deepstream, but tend to break down if multiple sources (specifically multiple ZED cameras) are used, which is a problem for the future.

[Myzhar]

The frame_id field is almost hidden for most of the users and if you think that it's better to remove it please feel free to submit a new PR. This is a feature for advanced users and only real tests on the field can justify its presence or not.