search

TheImagingSource / tiscamera

The Linux SDK for The Imaging Source cameras.
https://www.theimagingsource.com
Apache License 2.0
299 stars 148 forks source link

DFK 33UX273: Cannot set video/x-bayer, format=rggb #445

Closed lkucalaba closed 2 years ago

lkucalaba commented 2 years ago

Hello The Imaging Source, We are trying to set the camera we have (model DFK 33UX273) to this mode to achieve the highest possible framerate:

Available gstreamer-1.0 caps: video/x-bayer,format=rggb,width=1440,height=1080,framerate={2500000/10593,120/1,60/1,30/1,15/1,5/1,1/1};

We made this one change to the c example program 05-set-format.c on line 71:

    GstStructure* structure = gst_structure_from_string("video/x-bayer", NULL);
    gst_structure_set(structure,
                      "format", G_TYPE_STRING, "rggb",
                      "width", G_TYPE_INT, 1440,
                      "height", G_TYPE_INT, 1080,
                      "framerate", GST_TYPE_FRACTION, 2500000, 10593,
                      NULL);

Running the 05-set-format example program, this is the output we see on the console:

$ ./05-set-format 
Press enter to stop the stream.
0:00:00.220737200  5786 0x55dcebff7d40 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<tcamsrc-mainsrc:src> could not send sticky events
0:00:02.298346160  5786 0x55dcebff7d40 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<tcamsrc-mainsrc> error: Internal data stream error.
0:00:02.298393524  5786 0x55dcebff7d40 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<tcamsrc-mainsrc> error: streaming stopped, reason not-negotiated (-4)
^C

What is the proper way to set this camera in this mode?

Thanks, -Luke

TIS-Stefan commented 2 years ago

The reason is a "not negotiated", which means, one of the Gstreamer modules in your pipeline does not like "video/x-bayer". You may need to convert it using "bayer2rgb" to a format, that the following Gstreamer modules like. In the sample, it is "videoconvert". You may use

GstStructure* structure = gst_structure_from_string("video/x-raw", NULL);
    gst_structure_set(structure,
                      "format", G_TYPE_STRING, "BGRx",
                      "width", G_TYPE_INT, 1440,
                      "height", G_TYPE_INT, 1080,
                      "framerate", GST_TYPE_FRACTION, 2500000, 10593,
                      NULL);

in order to achieve the same result, but with maximum fps. The tcambin will debayer the incoming video format from GRAY8 to RGB automatically.

TIS-Stefan commented 2 years ago

I may should add, the format you specify and pass to the caps filter is not the format used by the camera. It will be the output format of the tcambin. The tcambin sets the best matching format in the camera, usually GRAY8 or GRAY16_LE and converts it to RGB, which is understood by videoconvert and xvimageinsk.

Stefan

lkucalaba commented 2 years ago

Thanks for the quick response Stefan. After further investigation and talking with other engineers on my team, we believe this may be a problem with my specific laptop configuration since now I am working from home (like many people these days). When I plug in a TIS camera and run tcam-capture, I am unable to change the pixel format or frame rate using the GUI (the combo box is enabled but it seems like there is only one choice populated). I cannot switch between cameras at runtime either. When I start the program I see a chooser dialog box to pick the TIS camera (I have 2 plugged in). On my desktop system I recall being able to change these settings and switch between cameras. Do you know why I would be able to change these settings with tcam-capture on a desktop machine but not on a laptop? Other than the hardware differences, they are basically identical OS (Ubuntu 18.04) and system configuration. My laptop is Intel but my desktop is AMD. -Luke

On Fri, Jan 14, 2022 at 10:06 AM TIS-Stefan @.***> wrote:

I may should add, the format you specify and pass to the caps filter is not the format used by the camera. It will be the output format of the tcambin. The tcambin sets the best matching format in the camera, usually GRAY8 or GRAY16_LE and converts it to RGB, which is understood by videoconvert and xvimageinsk.

Stefan

— Reply to this email directly, view it on GitHub https://github.com/TheImagingSource/tiscamera/issues/445#issuecomment-1013206552, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVNDTS4EWCWIWTIDWMUDZ5DUWA3XXANCNFSM5L66P3XQ . You are receiving this because you authored the thread.Message ID: @.***>

--

[image: Facebook] https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_dVUONoEmKyChwBoEuUwnyBXhYWqgP1SVtR2k_MPp4zzPbHIEta8XTaUC0Et-KgDBM=

[image: Twitter] https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_chDW18CE_jy1bfCHuOQFxymzD0ogf2QJjwO7wAI1ahxg==

[image: LinkedIn] https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_eZ_mBPwtDTE-UK6J6vdNE07T9DW5rBYtsJfkE-nqZvtpJRSSvhkgCklazS0OkMzaE=

Luke Kucalaba

Senior Software Engineer

M. 1.614.390.8404

P. 1.614.816.1991

528 Maier Pl

Columbus

OH

43215

path-robotics.com https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_dxEE-Y7Q4bGnEHbiIghZ1sEkvNRBwKrGRIs9gDfor2EQ==

-- This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.

TIS-Stefan commented 2 years ago

I am sorry to disappoint you, but it is not an issue of your laptop. Tcam-capture must list the formats rggb,rggb16, YUY2 and BGR in the format combobox. There are two resolutions: 1440x1080 and 640x480 and a set of frame rates.

If tcam-ctrl -c show shows all of them, tcam-capture must do too. The complete list on my computer is

Available gstreamer-1.0 caps:
video/x-bayer,format=rggb,width=1440,height=1080,framerate={2500000/10593,120/1,60/1,30/1,15/1,5/1,1/1};
video/x-bayer,format=rggb,width=640,height=480,framerate={601/1,480/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1};
video/x-bayer,format=rggb16,width=1440,height=1080,framerate={120/1,60/1,30/1,15/1,5/1,1/1};
video/x-bayer,format=rggb16,width=640,height=480,framerate={377/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1};
video/x-raw,format=YUY2,width=1440,height=1080,framerate={120/1,60/1,30/1,15/1,5/1,1/1};
video/x-raw,format=YUY2,width=640,height=480,framerate={596/1,480/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1};
video/x-raw,format=BGR,width=1440,height=1080,framerate={80/1,60/1,30/1,15/1,5/1,1/1};
video/x-raw,format=BGR,width=640,height=480,framerate={398/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1}

Also the c list formats sample must do so.

I cannot switch between cameras at runtime either. That is a tcam-capture issue, not a hardware issue.

I must admit, I have no idea, what went wrong. Please run lsusb -t The output should be:

/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 1: Dev 5, If 0, Class=Miscellaneous Device, Driver=, 5000M
        |__ Port 1: Dev 5, If 1, Class=Miscellaneous Device, Driver=, 5000M
        |__ Port 1: Dev 5, If 2, Class=Miscellaneous Device, Driver=, 5000M
        |__ Port 1: Dev 5, If 3, Class=Video, Driver=uvcvideo, 5000M
        |__ Port 1: Dev 5, If 4, Class=Video, Driver=, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
    |__ Port 10: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

The " Class=Video, Driver=uvcvideo, 5000M" is the important part. If there is "480" instead of "5000", then the camera thinks, it is connected to an USB 2.0 port. Then it does not work correctly. You may check that and the USB cable. (Oh... its a hardware issue then)

lkucalaba commented 2 years ago

I figured it out! When I run tcam-capture from the command line, it works perfectly. When I run using the GNOME desktop/application shortcut (from upper-left Activities menu), it doesn't work!! I will run from the command line from now on but want to report this problem with the desktop application shortcut -Luke

On Fri, Jan 14, 2022 at 11:38 AM TIS-Stefan @.***> wrote:

I am sorry to disappoint you, but it is not an issue of your laptop. Tcam-capture must list the formats rggb,rggb16, YUY2 and BGR in the format combobox. There are two resolutions: 1440x1080 and 640x480 and a set of frame rates.

If tcam-ctrl -c show shows all of them, tcam-capture must do too. The complete list on my computer is

Available gstreamer-1.0 caps: video/x-bayer,format=rggb,width=1440,height=1080,framerate={2500000/10593,120/1,60/1,30/1,15/1,5/1,1/1}; video/x-bayer,format=rggb,width=640,height=480,framerate={601/1,480/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1}; video/x-bayer,format=rggb16,width=1440,height=1080,framerate={120/1,60/1,30/1,15/1,5/1,1/1}; video/x-bayer,format=rggb16,width=640,height=480,framerate={377/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1}; video/x-raw,format=YUY2,width=1440,height=1080,framerate={120/1,60/1,30/1,15/1,5/1,1/1}; video/x-raw,format=YUY2,width=640,height=480,framerate={596/1,480/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1}; video/x-raw,format=BGR,width=1440,height=1080,framerate={80/1,60/1,30/1,15/1,5/1,1/1}; video/x-raw,format=BGR,width=640,height=480,framerate={398/1,5000000/20833,120/1,60/1,30/1,15/1,5/1,1/1}

Also the c list formats sample must do so.

I cannot switch between cameras at runtime either. That is a tcam-capture issue, not a hardware issue.

I must admit, I have no idea, what went wrong. Please run lsusb -t The output should be:

/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 5000M | Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M | Port 1: Dev 5, If 0, Class=Miscellaneous Device, Driver=, 5000M | Port 1: Dev 5, If 1, Class=Miscellaneous Device, Driver=, 5000M | Port 1: Dev 5, If 2, Class=Miscellaneous Device, Driver=, 5000M | Port 1: Dev 5, If 3, Class=Video, Driver=uvcvideo, 5000M | Port 1: Dev 5, If 4, Class=Video, Driver=, 5000M /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M |__ Port 10: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

The " Class=Video, Driver=uvcvideo, 5000M" is the important part. If there is "480" instead of "5000", then the camera thinks, it is connected to an USB 2.0 port. Then it does not work correctly. You may check that and the USB cable. (Oh... its a hardware issue then)

— Reply to this email directly, view it on GitHub https://github.com/TheImagingSource/tiscamera/issues/445#issuecomment-1013280001, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVNDTS44HYTCWBAYE6MNBZTUWBGPZANCNFSM5L66P3XQ . You are receiving this because you authored the thread.Message ID: @.***>

--

[image: Facebook] https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_dVUONoEmKyChwBoEuUwnyBXhYWqgP1SVtR2k_MPp4zzPbHIEta8XTaUC0Et-KgDBM=

[image: Twitter] https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_chDW18CE_jy1bfCHuOQFxymzD0ogf2QJjwO7wAI1ahxg==

[image: LinkedIn] https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_eZ_mBPwtDTE-UK6J6vdNE07T9DW5rBYtsJfkE-nqZvtpJRSSvhkgCklazS0OkMzaE=

Luke Kucalaba

Senior Software Engineer

M. 1.614.390.8404

P. 1.614.816.1991

528 Maier Pl

Columbus

OH

43215

path-robotics.com https://cloud.letsignit.com/collect/bc/616daeca87b9c7272d243335?p=cbAHb5-LHJ3xilU8LoeU0h6-czqywVIGMqoN1M5ZzFtPXgcwYJFHY2Sol3Wwl7foBI4E5IZ6o70QizAlJ_YabxWtslGjar55_jAKzJ-Ye_dxEE-Y7Q4bGnEHbiIghZ1sEkvNRBwKrGRIs9gDfor2EQ==

-- This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.

TIS-Stefan commented 2 years ago

Maybe after a restart of the computer all settings are system wide and it works from desktop shortcut too. But I am guessing only.

Stefan

lkucalaba commented 2 years ago

I've restarted many times, and reinstalled the tiscamera drivers many time. So the desktop vs command-line is one aspect or symptom of the problem. The real goal for us is to be able to set this mode using the command-line c example provided (05-set-format). This revelation doesn't change the fact that we can't set the mode via the sample application which is run on the command-line in a terminal.

Here is the lsusb info you requested: (I'm using a USB-3.0 hub)

$ lsusb
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 009: ID 199e:908f The Imaging Source Europe GmbH 
Bus 002 Device 002: ID 2109:0817 VIA Labs, Inc. 
Bus 002 Device 007: ID 199e:908f The Imaging Source Europe GmbH 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 046d:c31c Logitech, Inc. Keyboard K120
Bus 001 Device 009: ID 046d:c077 Logitech, Inc. M105 Optical Mouse
Bus 001 Device 012: ID 046d:0a8f Logitech, Inc. 
Bus 001 Device 008: ID 2109:2817 VIA Labs, Inc. 
Bus 001 Device 007: ID 8087:0aaa Intel Corp. 
Bus 001 Device 005: ID 5986:211c Acer, Inc 
Bus 001 Device 002: ID 1038:1122 SteelSeries ApS 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

$ lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 10000M
    |__ Port 2: Dev 7, If 4, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 7, If 2, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 7, If 0, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 7, If 3, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 7, If 1, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 3: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 2: Dev 9, If 0, Class=Miscellaneous Device, Driver=, 5000M
        |__ Port 2: Dev 9, If 1, Class=Miscellaneous Device, Driver=, 5000M
        |__ Port 2: Dev 9, If 2, Class=Miscellaneous Device, Driver=, 5000M
        |__ Port 2: Dev 9, If 3, Class=Video, Driver=uvcvideo, 5000M
        |__ Port 2: Dev 9, If 4, Class=Video, Driver=uvcvideo, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
    |__ Port 1: Dev 2, If 1, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 4: Dev 8, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 12, If 2, Class=Audio, Driver=snd-usb-audio, 12M
        |__ Port 1: Dev 12, If 0, Class=Audio, Driver=snd-usb-audio, 12M
        |__ Port 1: Dev 12, If 3, Class=Human Interface Device, Driver=usbhid, 12M
        |__ Port 1: Dev 12, If 1, Class=Audio, Driver=snd-usb-audio, 12M
        |__ Port 4: Dev 9, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 5: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 5: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 11: Dev 5, If 0, Class=Video, Driver=uvcvideo, 480M
    |__ Port 11: Dev 5, If 1, Class=Video, Driver=uvcvideo, 480M
    |__ Port 14: Dev 7, If 0, Class=Wireless, Driver=btusb, 12M
    |__ Port 14: Dev 7, If 1, Class=Wireless, Driver=btusb, 12M
lkucalaba commented 2 years ago

Increasing the frame rate to 2500000/10593 does work for BGRx mode.

I tried this alternate static method for rggb with same results (but new error message):

      GstElement* pipeline = gst_parse_launch("tcambin name=source ! video/x-bayer,format=rggb,width=1440,height=1080,framerate=2500000/10593 ! videoconvert ! ximagesink", &err);

$ ./05-set-format 
0:00:00.044015413 22312 0x563ad7011ad0 ERROR           GST_PIPELINE grammar.y:726:gst_parse_perform_link: could not link source to videoconvert0, videoconvert0 can't handle caps video/x-bayer, format=(string)rggb, width=(int)1440, height=(int)1080, framerate=(fraction)2500000/10593
Press enter to stop the stream.
0:00:00.240840830 22312 0x563ad7011590 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<tcamsrc-mainsrc> error: Internal data stream error.
0:00:00.240899133 22312 0x563ad7011590 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<tcamsrc-mainsrc> error: streaming stopped, reason not-linked (-1)
lkucalaba commented 2 years ago

I have new information about the problem with not being able to select resolution in the tcam-capture GUI tool. This seems to be a GUI rendering bug with GNOME/Ubuntu. It happens regardless of whether I run it from terminal or from GNOME app shortcut. I have a multi-monitor setup. On my primary laptop monitor, there is a major GUI rendering bug where all of the comboboxes only have one item in them. I can literally drag the window over to another monitor and the GUI bug appears magically fixed. I believe I have seen GUI issues like this with other applications so I think it's fair to say this is an OS problem.

One last thought on increasing the frame rate for BGRx mode. Although gstreamer allows you to set the framerate to 2500000/10593 (meaning, it doesn't complain), the observed FPS we are seeing in our tests actually tops out around 80 fps. Is this because the caps dump from "tcam-ctrl -c" shows max 80/1 fps for that BGRx mode?

video/x-raw,format=BGR,width=1440,height=1080,framerate={80/1,60/1,30/1,15/1,5/1,1/1};

Or maybe it's because we are enabling trigger mode with software triggering? We have a software trigger in the loop which may be limiting the performance. Is there any reason why we wouldn't be able to achieve something closer to the maximum frame rate (240fps) with trigger mode enabled and using software triggering?

I did some more digging into tcam-capture, looking at the python source code to try to figure out why that program seems to work to set the rggb mode but the c examples provided do not work for us. There is an interesting block of code in TcamView.py on the 'play' event handler here: https://github.com/TheImagingSource/tiscamera/blob/031c7762d785f457475d3ee8126fb358170f2e65/tools/tcam-capture/tcam_capture/TcamView.py#L269

        if self.video_format is not None:
            log.info("Setting format to {}".format(video_format))
            caps = self.pipeline.get_by_name("bin")
            caps.set_property("device-caps",
                              video_format)

I'd never seen that "device-caps" property before. So... I checked the tiscamera docs again and I saw this properties table: https://www.theimagingsource.com/documentation/tiscamera/tcam-gstreamer.html#tcambin

The table lists 'device-caps' but not 'caps'. Apparently 'caps' is a 'capsfilter' whereas 'device-caps' is string property of the tcambin plugin? Is there any advantage to using device-caps over capsfilter? I just want to set the mode to video/x-bayer,format=rggb at max framerate and framesize. Thanks

lkucalaba commented 2 years ago

Final update: I have video/x-bayer,format=rggb working now! Apparently I had to remove 'videoconvert' from the pipeline launch string. And now everything works as expected by setting the GstStructure in the capsfilter.

Then I stumbled upon this page: https://theimagingsource.deskpro.com/en/kb/articles/linux-set-video-formats-in-camera-and-memory

Bayer Raw 8 bit: format="video/x-bayer, format=rggb, width=640, height=480,framerate=30/1" displayformat="video/x-raw, format=GRAY8, width=640, height=480,framerate=30/1" gst-launch-1.0 tcambin ! $format ! capssetter join=false replace=true caps="$displayformat" ! videoconvert ! videoscale ! ximagesink

Is there an advantage to using this latter approach with the capssetter and the videoconvert and the displayformat and videoscale plugins (seems like a lot of extra overhead), or can i just remove the videoconvert plugin and use the former approach with the GstStructure and the capsfilter?

Thanks, -Luke

TIS-Stefan commented 2 years ago

Hello

Using video/x-bayer you will receive Bayer raw data, no colored images. If the image is save compressed, e.g. as jpeg, the color information is completely lost. It will look grayscale. You are right, videoconvert does not digest bayer raw formats. But videoconvert is necessary, if you want to display the image on screen.

Is there an advantage to using this latter approach with the capssetter and the videoconvert and the displayformat and videoscale plugins (seems like a lot of extra overhead), or can i just remove the videoconvert plugin and use the former approach with the GstStructure and the capsfilter

No, because the tcambin does this automatically for you, if you use "video/x-raw, format=BGRx".

Stefan

TIS-Stefan commented 2 years ago

Hello

Just an addition: The videoscale (https://gstreamer.freedesktop.org/documentation/videoscale/index.html) is not needed. It is used in the sample to scale down the video for display. However, simply use the tcambin with video/x-raw, format=BGRx and set the high frame rate. That should do the job.

Stefan