Make jetson_utils work with gstreamer and yolov8 api

[samthephantom]

Hi dusty. I used to decode h.264-codec rtsp camera stream with gstreamer python api ( I built the pipeline with nvv4l2decoder and nvvidconv). Here is the detail:

1. The pipeline convert image from NVMM to CPU RAM with nvvidconv
2. I get the decoded image data from appsink by sink.emit("pull-sample").

3. The image data is read and transferred to numpy like this:

   
   def new_buffer(sink, data):
   # pull sample
   sample = sink.emit("pull-sample")
   buf = sample.get_buffer()
   caps = sample.get_caps()
   
   # create buffer
   arr = np.ndarray(
       (caps.get_structure(0).get_value('height'),
        caps.get_structure(0).get_value('width'),
        4),
       buffer=buf.extract_dup(0, buf.get_size()),
       dtype=np.uint8)
   
   # convert BGRx to RGB for yolov8 inference
   data.img = cv2.cvtColor(arr, cv2.COLOR_BGRA2BGR)
   
   data.data_valid()
   data.frame_shape = data.img.shape
   data.frame_timestamp = time.time()
   
   if data.frame < 10000:
       data.frame = data.frame+1
   else:
       data.frame = 0
   return Gst.FlowReturn.OK

Please share your experience on how to make cudaImage work with gstreamer, thanks. And I couldn't find out if the yolov8 python api   accept the cudaImage as input.

[johnnynunez]

Hi dusty. I used to decode h.264-codec rtsp camera stream with gstreamer python api ( I built the pipeline with nvv4l2decoder and nvvidconv). Here is the detail:

1. The pipeline convert image from NVMM to CPU RAM with nvvidconv
2. I get the decoded image data from appsink by sink.emit("pull-sample").
3. The image data is read and transferred to numpy like this:

def new_buffer(sink, data):
    # pull sample
    sample = sink.emit("pull-sample")
    buf = sample.get_buffer()
    caps = sample.get_caps()

    # create buffer
    arr = np.ndarray(
        (caps.get_structure(0).get_value('height'),
         caps.get_structure(0).get_value('width'),
         4),
        buffer=buf.extract_dup(0, buf.get_size()),
        dtype=np.uint8)

    # convert BGRx to RGB for yolov8 inference
    data.img = cv2.cvtColor(arr, cv2.COLOR_BGRA2BGR)

    data.data_valid()
    data.frame_shape = data.img.shape
    data.frame_timestamp = time.time()

    if data.frame < 10000:
        data.frame = data.frame+1
    else:
        data.frame = 0
    return Gst.FlowReturn.OK

Please share your experience on how to make cudaImage work with gstreamer, thanks. And I couldn't find out if the yolov8 python api accept the cudaImage as input.

https://github.com/STCE-Detector/small-fast-detector/blob/main/tracker/track_recognize.py