using fbdevsink (was: -r not working with fbdevsink, solved)

[Boaztheostrich]

I was trying to rotate the video output using this command:

uxplay -vs fbdevsink -r L

(-r R also didn't work)

But it would not rotate, I am using a Pi CM4 with the following carrier board: https://www.fysetc.com/products/raspberry-pi-cm4-hmi-display-module-small-and-high-resolution-him-dpi-interface-capacitive-touch-screen-module

Using an unpatched version of uxplay with everything up to date.

I've attached some picture below to showcase the issue:

Additionally the image output stays over the terminal, or desktop if you run this command with desktop, until you run the clear command.

Thanks, Boaz

[Boaztheostrich]

It also has an invisible "box" around it, meaning that:

uxplay -vs fbdevsink -s 800x480

uxplay -vs fbdevsink -s 1167x800

and

uxplay -vs fbdevsink -s 1000x600

all have the same output on the screen, shown below:

If you go larger though, such as

uxplay -vs fbdevsink -s 1667x1000

It does increase the size of the output to where it is bigger than the 800x480 screen, as shown below:

Thanks

[fduncanh]

did you try kmssink?

[fduncanh]

tested on R Pi OS lite Bookworm.

I confirmed that uxplay -vs dfbvideosink works (without -r R), but leaves parts of image on screen when window closes.

• with the option "-r R" it fails to work (cannot build working pipeline)

use -vs kmssink instead.

[Boaztheostrich]

This is what I get for kmssink

boazburnett@raspberrypi:~ $ uxplay -vs kmssink
UxPlay 1.62: An Open-Source AirPlay mirroring and audio-streaming server.
Failed to initialize GStreamer video renderer
using system MAC address e4:5f:01:fa:db:68
Initialized server socket(s)
GStreamer error: Could not open DRM module (NULL)
Failed to initialize GStreamer video renderer
Initialized server socket(s)
GStreamer error: Could not open DRM module (NULL)

[fduncanh]

does "gst-inspect-1.0 kmssink" show it is present?

[Boaztheostrich]

It does!

boazburnett@raspberrypi:~ $ gst-inspect-1.0 kmssink
Factory Details:
  Rank                     secondary (128)
  Long-name                KMS video sink
  Klass                    Sink/Video
  Description              Video sink using the Linux kernel mode setting API
  Author                   Víctor Jáquez <vjaquez@igalia.com>
  Documentation            https://gstreamer.freedesktop.org/documentation/kms/#kmssink-page

Plugin Details:
  Name                     kms
  Description              Video sink using the Linux kernel mode setting API
  Filename                 /usr/local/lib/aarch64-linux-gnu/gstreamer-1.0/libgstkms.so
  Version                  1.22.12
  License                  LGPL
  Source module            gst-plugins-bad
  Documentation            https://gstreamer.freedesktop.org/documentation/kms/
  Source release date      2024-04-29
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstKMSSink

Implemented Interfaces:
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)BGRA, (string)RGBA, (string)P016_LE, (string)P010_10LE, (string)NV24, (string)BGRx, (string)RGBx, (string)RGB, (string)BGR, (string)Y42B, (string)NV61, (string)NV16, (string)UYVY, (string)YVYU, (string)YUY2, (string)I420, (string)YV12, (string)NV21, (string)NV12, (string)RGB16, (string)BGR16 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:

  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true

  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 

  bus-id              : DRM bus ID
                        flags: readable, writable
                        String. Default: null

  can-scale           : User can tell kmssink if the driver can support scale
                        flags: readable, writable
                        Boolean. Default: true

  connector-id        : DRM connector id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
:

[fduncanh]

what about "lsmod | grep drm",

[Boaztheostrich]

That returns the following:

boazburnett@raspberrypi:~ $ lsmod | grep drm
drm                   688128  0
drm_panel_orientation_quirks    24576  1 drm
backlight              24576  2 gpio_backlight,drm

I think this could be related to my config.txt file and specifically the vc4-kms-3d driver being disabled.

# For more options and information see
# http://rptl.io/configtxt
# Some settings may impact device functionality. See link above for details

# Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

# Additional overlays and parameters are documented
# /boot/firmware/overlays/README

# Automatically load overlays for detected cameras
camera_auto_detect=1

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Automatically load initramfs files, if found
auto_initramfs=1

# Enable DRM VC4 V3D driver
#dtoverlay=vc4-kms-v3d
#max_framebuffers=2

# stuff for the screen
dtoverlay=ctp40,touchscreen-swapped-x-y,touchscreen-inverted-y
enable_dpi_lcd=1
dpi_group=2
dpi_mode=87
dpi_output_format=0x7f216
dpi_timings=480 0 10 16 59 800 0 15 113 15 0 0 0 60 0 32000000 6

display_lcd_rotate=0 # 1 is horizontal

# Don't have the firmware create an initial video= setting in cmdline.txt.
# Use the kernel's default instead.
disable_fw_kms_setup=1

# Run in 64-bit mode
arm_64bit=1

# Disable compensation for displays with overscan
disable_overscan=1

# Run as fast as firmware / board allows
arm_boost=1

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

[all]

The issue is if I enable it I can't get the screen to boot

[fduncanh]

complicated! not sure what you can do.

[Boaztheostrich]

Very complicated haha, and I've been unable to get any real help from the manufacturer. I may have to learn how to make a video driver to replace the existing one. idk

On the bright side this:

boazburnett@raspberrypi:~ $ uxplay -vs "videoscale ! video/x-raw,width=480,height=800 ! fbdevsink"

Is full screen and works better than any other command I have tried on this display!

The latency is still not what I want it to be but it is getting there.

Thanks again for all of the help!!

If you have any suggestions for further optimizing the above command it would be appreciated, it looks like there are a lot of gstreamer options I can play with

[fduncanh]

If you need the rotation, you should use GST_DEBUG to find out why it does not work. maybe moving the position in the pipeline where the rotation is done might solve it (see renderers/video_renderer_gstreamer.c)

https://github.com/FDH2/UxPlay/blob/master/renderers/video_renderer_gstreamer.c#L150

closing issue for now

[fduncanh]

PS you should test your gstreamer pipeline directly, without uxplay:

works: gst-launch-1.0 -v videotestsrc ! videoscale ! fbdevsink

doesnt work : (find out why?)

gst-launch-1.0 -v videotestsrc ! videoscale ! videoflip video-direction=GST_VIDEO_ORIENTATION_90R ! fbdevsink

works:

gst-launch-1.0 -v videotestsrc ! videoscale ! videoflip video-direction=GST_VIDEO_ORIENTATION_90R ! kmssink

[fduncanh]

OK what works is

gst-launch-1.0 -v videotestsrc !  videoflip video-direction=GST_VIDEO_ORIENTATION_90R ! videoconvert !videoscale! fbdevsink

https://gstreamer.freedesktop.org/documentation/videofilter/videoflip.html?gi-language=c

https://gstreamer.freedesktop.org/documentation/videoconvertscale/videoconvert.html?gi-language=c

https://gstreamer.freedesktop.org/documentation/videoconvertscale/videoscale.html?gi-language=c

[fduncanh]

I will see if UxPlay should have a ...! videoconvert ! .. in its pipeline (or ! videoconvertscale !)

https://gstreamer.freedesktop.org/documentation/videoconvertscale/videoconvertscale.html?gi-language=c

[fduncanh]

There is still the issue that fbdevsink leaves stuff behind when it closes.

[fduncanh]

so;

uxplay -r R -vs " videoconvertscale ! fbdevsink"

seems to do the job. (apart from the screen reset on closing)

[Boaztheostrich]

Thank you!

[fduncanh]

the github uxplay source has just been updated to move videoconvert (which was already there, but before videoflip) to after it, and then to add a videoscale before the videosink.

• now just "uxplay -vs fbdevsink -r R" should work without needing to add videoscale or videconvert

(or for you, uxplay -vs "video/x-raw,width=480,height=800 ! fbdevsink" -r R )

[Boaztheostrich]

Excellent! Thanks again for the help