search

clydemcqueen / gscam2

ROS2 camera driver for GStreamer-based video streams, supports intra-process communication
72 stars 13 forks source link

gscam2 pipeline with udp source on an Nvidia Jetson #42

Closed jmachuca77 closed 4 months ago

jmachuca77 commented 4 months ago

I need a bit of help with the pipeline string, I have a udp camera connected and I can visualize the video with this gst-launch-1.0 command:

gst-launch-1.0 udpsrc port=50008 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! fpsdisplaysink

Im trying to use that as an input to gscam by setting the following env variable:

export GSCAM_CONFIG='udpsrc port=50008 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv"

However I get the following error:

[FATAL] [1715812318.234746933] [gscam_publisher]: Cannot link outelement("nvvconv0") -> sink

I tried with a simple pipeline with 'videotestsrc ! video/x-raw, format=BGRx ! videoconvert' and that works, but not with the above settings. Any ideas on what is wrong? the following is the full output:

$ export GSCAM_CONFIG='udpsrc port=50008 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv'

$ ros2 run gscam2 gscam_main --ros-args -p camera_info_url:=file://$PWD/my_camera.ini
[INFO] [1715812631.997915183] [gscam_publisher]: use_intra_process_comms=0
[INFO] [1715812631.998675774] [gscam_publisher]: gst_plugin_path = 
[INFO] [1715812631.998721311] [gscam_publisher]: gscam_config = 
[INFO] [1715812631.998743583] [gscam_publisher]: sync_sink = true
[INFO] [1715812631.998759232] [gscam_publisher]: preroll = false
[INFO] [1715812631.998773184] [gscam_publisher]: use_gst_timestamps = false
[INFO] [1715812631.998790176] [gscam_publisher]: image_encoding = rgb8
[INFO] [1715812631.998806081] [gscam_publisher]: camera_info_url = file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715812631.998822785] [gscam_publisher]: camera_name = 
[INFO] [1715812631.998838017] [gscam_publisher]: frame_id = camera_frame
[INFO] [1715812631.998857410] [gscam_publisher]: Using GSCAM_CONFIG env var: udpsrc port=50008 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv
[INFO] [1715812631.998914723] [gscam_publisher]: camera calibration URL: file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715812631.999182792] [gscam_publisher]: Loaded camera calibration from file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715812632.011161754] [gscam_publisher]: Gstreamer initialized
[INFO] [1715812632.011244668] [gscam_publisher]: Gstreamer version: GStreamer 1.16.3
[FATAL] [1715812632.069449652] [gscam_publisher]: Cannot link outelement("nvvconv0") -> sink

[INFO] [1715812632.069807291] [gscam_publisher]: Pipeline deleted
clydemcqueen commented 4 months ago

The nvvidconv element isn't linking to the appsink element. This might be caused by a pipeline negotiation failure. In this case appsink is expecting "video/x-raw,format=(string)RGB", can nvvidconv provide this?

Ptr to caps in code: https://github.com/clydemcqueen/gscam2/blob/16cbefb1cbd1a86e0803827dda0977d63694cf89/src/gscam_node.cpp#L162

You can increase debugging to see what is going on: export GST_DEBUG=3. More info on debugging: https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html?gi-language=c

jmachuca77 commented 4 months ago

Yeah I figured out that nvvidconv doesn't look like it supports that format which is very unfortunate:

$ gst-inspect-1.0 nvvidconv
Factory Details:
  Rank                     primary (256)
  Long-name                NvVidConv Plugin
  Klass                    Filter/Converter/Video/Scaler
  Description              Converts video from one colorspace to another & Resizes
  Author                   amit pandya <apandya@nvidia.com>

Plugin Details:
  Name                     nvvidconv
  Description              video Colorspace conversion & scaler
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvidconv.so
  Version                  1.2.3
  License                  Proprietary
  Source module            gstreamer-nvvconv-plugin
  Binary package           GStreamer nvvconv Plugin
  Origin URL               http://nvidia.com/

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseTransform
                         +----Gstnvvconv

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { (string)I420, (string)I420_10LE, (string)P010_10LE, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV12, (string)NV16, (string)NV24, (string)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B, (string)Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)I420, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV12, (string)NV16, (string)NV24, (string)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B, (string)Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { (string)I420, (string)I420_10LE, (string)P010_10LE, (string)I420_12LE, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV12, (string)NV16, (string)NV24, (string)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B, (string)Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)I420, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV12, (string)NV16, (string)NV24, (string)P010_10LE, (string)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B, (string)Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  bl-output           : Blocklinear output, applicable only for memory:NVMM NV12 format output buffer when compute hw is VIC
                        flags: readable, writable
                        Boolean. Default: true
  bottom              : Pixels to crop at bottom
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  compute-hw          : Compute Scaling HW
                        flags: readable, writable, controllable
                        Enum "GstNvVidConvComputeHWType" Default: 0, "Default"
                           (0): Default          - Default, VIC for Jetson
                           (1): GPU              - GPU
                           (2): VIC              - VIC
  flip-method         : video flip methods
                        flags: readable, writable, controllable
                        Enum "GstNvVideoFlipMethod" Default: 0, "none"
                           (0): none             - Identity (no rotation)
                           (1): counterclockwise - Rotate counter-clockwise 90 degrees
                           (2): rotate-180       - Rotate 180 degrees
                           (3): clockwise        - Rotate clockwise 90 degrees
                           (4): horizontal-flip  - Flip horizontally
                           (5): upper-right-diagonal - Flip across upper right/lower left diagonal
                           (6): vertical-flip    - Flip vertically
                           (7): upper-left-diagonal - Flip across upper left/lower right diagonal
  gpu-id              : Set GPU Device ID for operation
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  interpolation-method: Set interpolation methods
                        flags: readable, writable, controllable
                        Enum "GstInterpolationMethod" Default: 0, "Nearest"
                           (0): Nearest          - Nearest
                           (1): Bilinear         - Bilinear
                           (2): 5-Tap            - 5-Tap
                           (3): 10-Tap           - 10-Tap
                           (4): Smart            - Smart
                           (5): Nicest           - Nicest
  left                : Pixels to crop at left
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "nvvconv0"
  nvbuf-memory-type   : Type of NvBufSurface Memory to be allocated for output buffers when compute hw is GPU and memory is NVMM
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstNvVidConvBufMemoryType" Default: 0, "nvbuf-mem-default"
                           (0): nvbuf-mem-default - Default memory allocated, specific to particular platform
                           (1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory
                           (2): nvbuf-mem-cuda-device - Allocate Device cuda memory
                           (4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
  output-buffers      : number of output buffers
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 1 - 4294967295 Default: 4 
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  qos                 : Handle Quality-of-Service events
                        flags: readable, writable
                        Boolean. Default: false
  right               : Pixels to crop at right
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  silent              : Produce verbose output ?
                        flags: readable, writable
                        Boolean. Default: false
  top                 : Pixels to crop at top
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0

However videoconvert does, so I switched to that but Im still getting the same error, with no other output even increasing the debug level for GST to 3, I don't see RGB8 listed though so could it be that the source is in RGB8 and that's why its failing? and if so what would be a workaround?:

Factory Details:
  Rank                     none (0)
  Long-name                Colorspace converter
  Klass                    Filter/Converter/Video
  Description              Converts video from one colorspace to another
  Author                   GStreamer maintainers <gstreamer-devel@lists.freedesktop.org>

Plugin Details:
  Name                     videoconvert
  Description              Colorspace conversion
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideoconvert.so
  Version                  1.16.3
  License                  LGPL
  Source module            gst-plugins-base
  Source release date      2020-10-21
  Binary package           GStreamer Base Plugins (Ubuntu)
  Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-base1.0

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseTransform
                         +----GstVideoFilter
                               +----GstVideoConvert

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)I420, (string)YV12, (string)YUY2, (string)UYVY, (string)AYUV, (string)VUYA, (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB, (string)ABGR, **(string)RGB**, (string)BGR, (string)Y41B, (string)Y42B, (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)Y210, (string)Y410, (string)NV12, (string)NV21, (string)GRAY8, (string)GRAY16_BE, (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16, (string)RGB15, (string)BGR15, (string)UYVP, (string)A420, (string)RGB8P, (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64, (string)AYUV64, (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE, (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR, (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24, (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE, (string)A422_10BE, (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61, (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY, (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE, (string)BGR10A2_LE, (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE, (string)I420_12BE, (string)I420_12LE, (string)I422_12BE, (string)I422_12LE, (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32, (string)NV12_10LE32, (string)NV16_10LE32, (string)NV12_10LE40 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(ANY)
                 format: { (string)I420, (string)YV12, (string)YUY2, (string)UYVY, (string)AYUV, (string)VUYA, (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB, (string)ABGR, **(string)RGB**, (string)BGR, (string)Y41B, (string)Y42B, (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)Y210, (string)Y410, (string)NV12, (string)NV21, (string)GRAY8, (string)GRAY16_BE, (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16, (string)RGB15, (string)BGR15, (string)UYVP, (string)A420, (string)RGB8P, (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64, (string)AYUV64, (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE, (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR, (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24, (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE, (string)A422_10BE, (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61, (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY, (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE, (string)BGR10A2_LE, (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE, (string)I420_12BE, (string)I420_12LE, (string)I422_12BE, (string)I422_12LE, (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32, (string)NV12_10LE32, (string)NV16_10LE32, (string)NV12_10LE40 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)I420, (string)YV12, (string)YUY2, (string)UYVY, (string)AYUV, (string)VUYA, (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB, (string)ABGR, **(string)RGB**, (string)BGR, (string)Y41B, (string)Y42B, (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)Y210, (string)Y410, (string)NV12, (string)NV21, (string)GRAY8, (string)GRAY16_BE, (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16, (string)RGB15, (string)BGR15, (string)UYVP, (string)A420, (string)RGB8P, (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64, (string)AYUV64, (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE, (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR, (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24, (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE, (string)A422_10BE, (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61, (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY, (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE, (string)BGR10A2_LE, (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE, (string)I420_12BE, (string)I420_12LE, (string)I422_12BE, (string)I422_12LE, (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32, (string)NV12_10LE32, (string)NV16_10LE32, (string)NV12_10LE40 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(ANY)
                 format: { (string)I420, (string)YV12, (string)YUY2, (string)UYVY, (string)AYUV, (string)VUYA, (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB, (string)ABGR, **(string)RGB**, (string)BGR, (string)Y41B, (string)Y42B, (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)Y210, (string)Y410, (string)NV12, (string)NV21, (string)GRAY8, (string)GRAY16_BE, (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16, (string)RGB15, (string)BGR15, (string)UYVP, (string)A420, (string)RGB8P, (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64, (string)AYUV64, (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE, (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR, (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24, (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE, (string)A422_10BE, (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61, (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY, (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE, (string)BGR10A2_LE, (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE, (string)I420_12BE, (string)I420_12LE, (string)I422_12BE, (string)I422_12LE, (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32, (string)NV12_10LE32, (string)NV16_10LE32, (string)NV12_10LE40 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  alpha-mode          : Alpha Mode to use
                        flags: readable, writable
                        Enum "GstVideoAlphaMode" Default: 0, "copy"
                           (0): copy             - GST_VIDEO_ALPHA_MODE_COPY
                           (1): set              - GST_VIDEO_ALPHA_MODE_SET
                           (2): mult             - GST_VIDEO_ALPHA_MODE_MULT
  alpha-value         : Alpha Value to use
                        flags: readable, writable
                        Double. Range:               0 -               1 Default:               1 
  chroma-mode         : Chroma Resampling Mode
                        flags: readable, writable
                        Enum "GstVideoChromaMode" Default: 0, "full"
                           (0): full             - GST_VIDEO_CHROMA_MODE_FULL
                           (1): upsample-only    - GST_VIDEO_CHROMA_MODE_UPSAMPLE_ONLY
                           (2): downsample-only  - GST_VIDEO_CHROMA_MODE_DOWNSAMPLE_ONLY
                           (3): none             - GST_VIDEO_CHROMA_MODE_NONE
  chroma-resampler    : Chroma resampler method
                        flags: readable, writable
                        Enum "GstVideoResamplerMethod" Default: 1, "linear"
                           (0): nearest          - GST_VIDEO_RESAMPLER_METHOD_NEAREST
                           (1): linear           - GST_VIDEO_RESAMPLER_METHOD_LINEAR
                           (2): cubic            - GST_VIDEO_RESAMPLER_METHOD_CUBIC
                           (3): sinc             - GST_VIDEO_RESAMPLER_METHOD_SINC
                           (4): lanczos          - GST_VIDEO_RESAMPLER_METHOD_LANCZOS
  dither              : Apply dithering while converting
                        flags: readable, writable
                        Enum "GstVideoDitherMethod" Default: 4, "bayer"
                           (0): none             - GST_VIDEO_DITHER_NONE
                           (1): verterr          - GST_VIDEO_DITHER_VERTERR
                           (2): floyd-steinberg  - GST_VIDEO_DITHER_FLOYD_STEINBERG
                           (3): sierra-lite      - GST_VIDEO_DITHER_SIERRA_LITE
                           (4): bayer            - GST_VIDEO_DITHER_BAYER
  dither-quantization : Quantizer to use
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 1 
  gamma-mode          : Gamma Conversion Mode
                        flags: readable, writable
                        Enum "GstVideoGammaMode" Default: 0, "none"
                           (0): none             - GST_VIDEO_GAMMA_MODE_NONE
                           (1): remap            - GST_VIDEO_GAMMA_MODE_REMAP
  matrix-mode         : Matrix Conversion Mode
                        flags: readable, writable
                        Enum "GstVideoMatrixMode" Default: 0, "full"
                           (0): full             - GST_VIDEO_MATRIX_MODE_FULL
                           (1): input-only       - GST_VIDEO_MATRIX_MODE_INPUT_ONLY
                           (2): output-only      - GST_VIDEO_MATRIX_MODE_OUTPUT_ONLY
                           (3): none             - GST_VIDEO_MATRIX_MODE_NONE
  n-threads           : Maximum number of threads to use
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 1 
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "videoconvert0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  primaries-mode      : Primaries Conversion Mode
                        flags: readable, writable
                        Enum "GstVideoPrimariesMode" Default: 0, "none"
                           (0): none             - GST_VIDEO_PRIMARIES_MODE_NONE
                           (1): merge-only       - GST_VIDEO_PRIMARIES_MODE_MERGE_ONLY
                           (2): fast             - GST_VIDEO_PRIMARIES_MODE_FAST
  qos                 : Handle Quality-of-Service events
                        flags: readable, writable
                        Boolean. Default: true

output:

$ export GSCAM_CONFIG='udpsrc port=50008 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! queue ! videoconvert'

() $ ros2 run gscam2 gscam_main --ros-args -p camera_info_url:=file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715969737.881941999] [gscam_publisher]: use_intra_process_comms=0
[INFO] [1715969737.882382840] [gscam_publisher]: gst_plugin_path = 
[INFO] [1715969737.882418073] [gscam_publisher]: gscam_config = 
[INFO] [1715969737.882440026] [gscam_publisher]: sync_sink = true
[INFO] [1715969737.882456282] [gscam_publisher]: preroll = false
[INFO] [1715969737.882471642] [gscam_publisher]: use_gst_timestamps = false
[INFO] [1715969737.882491099] [gscam_publisher]: image_encoding = rgb8
[INFO] [1715969737.882508635] [gscam_publisher]: camera_info_url = file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715969737.882525787] [gscam_publisher]: camera_name = 
[INFO] [1715969737.882541660] [gscam_publisher]: frame_id = camera_frame
[INFO] [1715969737.882561340] [gscam_publisher]: Using GSCAM_CONFIG env var: udpsrc port=50008 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! queue ! videoconvert
[INFO] [1715969737.882613213] [gscam_publisher]: camera calibration URL: file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715969737.882868162] [gscam_publisher]: Loaded camera calibration from file:///home/jaime/kaya_ws/src/gscam2/cfg/my_camera.ini
[INFO] [1715969737.894341967] [gscam_publisher]: Gstreamer initialized
[INFO] [1715969737.894442577] [gscam_publisher]: Gstreamer version: GStreamer 1.16.3
[INFO] [1715969737.951642186] [gscam_publisher]: Adding RGB Caps
[FATAL] [1715969737.953675220] [gscam_publisher]: Cannot link outelement("videoconvert0") -> sink

[INFO] [1715969737.953955257] [gscam_publisher]: Pipeline deleted
^C^C[INFO] [1715969745.593488357] [rclcpp]: signal_handler(signum=2)
clydemcqueen commented 4 months ago

A few possibilities, easiest to hardest:

  1. Try ending with "! nvvidconv ! videoconvert".

  2. gscam uses image_transport, gscam2 does not. IIRC image_transport will do some format conversions. You might try gscam to see if it will behave differently.

  3. nvvidconv can provide YUY2, which maps to yuv422_yuy2 in ROS. Try setting the image_encoding to yuv422_yuy2. But the downstream nodes will need to convert yuv422_yuy2 to whatever you need.

  4. You could add more gstreamer formats to gscam2.

I am puzzled by something: I am running Ubuntu 22.04, which has gstreamer 1.20.1 installed (the default). When I run gst-inspect-1.0 videoconvert I see that RGB is listed in the SRC pad template format list, which I expected. You seem to be running gstreamer 1.16.3, and I don't see RGB listed in the SRC pad template format list. But it still works? I am not sure what to make of this.

P.S. setting GST_DEBUG=4 will give you way too much output, but buried in that you should see the create of caps objects, something like GST_EVENT gstevent.c:892:gst_event_new_caps: creating caps event video/x-raw, format=(string)YV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive

jmachuca77 commented 4 months ago

1 didn't work, I switched to gscam and that is working (I didn't go this way initially because I thought gscam was ros1 only). Didn't get to 3 and 4.

Closing for now thanks for your help. For future references I was not able to get gscam2 working, but was able to get gscam ros2 branch to work.

clydemcqueen commented 4 months ago

I'm glad gscam worked!