Fedora + niri - Failed to initialize pipeline state to playing - Feature request: support dmabufs

[thepragmaticmero]

Affected version 2.3.0 flatpak

Bug summary "Failed to start recording" Caused by: 0: Failed to initialize pipeline state to playing 1: Element failed to change its state

Steps to reproduce

• Click record
• Select screen
• Press share and get error

Expected behavior The regular Gnome behaviour, I'm using a smithay based window manager, "niri". OBS works fine on niri. But Kooha spits the previously shown error.

Relevant logs, screenshots, screencasts, etc. when running debugging command

$ RUST_BACKTRACE=1 RUST_LOG=kooha=debug GST_DEBUG=3 PIPEWIRE_DEBUG=3 flatpak run io.github.seadve.Kooha
0:00:00.036192301     3 0x564ce950cc90 WARN              vaapiblend gstvaapiblend.c:184:gst_vaapi_blend_initialize:<vaapiblend0> VPP does not support global alpha blending
2024-06-19T15:08:38.850995Z  INFO kooha::application: Kooha (io.github.seadve.Kooha)
2024-06-19T15:08:38.851011Z  INFO kooha::application: Version: 2.3.0 ()
2024-06-19T15:08:38.851016Z  INFO kooha::application: Datadir: /app/share/kooha
2024-06-19T15:08:41.421930Z DEBUG kooha::recording: Created screencast proxy version=Ok(5) available_cursor_modes=Ok(CursorMode(HIDDEN | EMBEDDED | METADATA)) available_source_types=Ok(SourceType(MONITOR | WINDOW | VIRTUAL))
2024-06-19T15:08:41.425209Z DEBUG kooha::recording: Selecting sources cursor_mode=CursorMode(EMBEDDED) source_type=SourceType(MONITOR) is_multiple_sources=false restore_token="" persist_mode=Persistent
2024-06-19T15:08:41.428313Z DEBUG kooha::recording: Starting session window_identifier=wayland:2chgoWjlimMWiVGYTfROIRHePh52ix4H
2024-06-19T15:08:43.581950Z DEBUG kooha::pipeline: file_path=/home/user/vid/Kooha-2024-06-19-10-08-43.mp4 framerate=Fraction(Ratio { numer: 60, denom: 1 }) profile="mp4" fd=17 stream_len=1 streams=[Stream { node_id: 78, id: Some("0"), position: Some((0, 0)), size: Some((1920, 1080)), source_type: Some(SourceType(MONITOR)) }] record_desktop_audio=true record_microphone=false select_area_data=None
[I][13443.082808] pw.context   | [      pipewire.c:  647 pw_init()] version 0.3.83
2024-06-19T15:08:43.592211Z DEBUG kooha::device: Finding device name for class `Sink`
2024-06-19T15:08:43.592225Z DEBUG kooha::device: Skipping device `pulsedevice0` as it has unknown device class `Audio/Source`
2024-06-19T15:08:43.592237Z DEBUG kooha::device: Skipping device `pulsedevice1` as it has unknown device class `Audio/Source`
2024-06-19T15:08:43.592243Z DEBUG kooha::device: Skipping device `pulsedevice2` as it has unknown device class `Audio/Source`
2024-06-19T15:08:43.592273Z DEBUG kooha::device: Skipping device `pulsedevice3` as it is not the default
2024-06-19T15:08:43.592536Z DEBUG kooha::pipeline: Found desktop audio with name `alsa_output.usb-Corsair_Corsair_HS60_PRO_Surround_USB_Sound_Adapter_00000000-00.analog-stereo.monitor`
0:00:04.811023886     2 0x55d725f41770 WARN                filesink gstfilesink.c:458:gst_file_sink_open_file:<filesink0> error: Could not open file "/home/user/vid/Kooha-2024-06-19-10-08-43.mp4" for writing.
0:00:04.811035729     2 0x55d725f41770 WARN                filesink gstfilesink.c:458:gst_file_sink_open_file:<filesink0> error: system error: No such file or directory
0:00:04.811050837     2 0x55d725f41770 WARN                basesink gstbasesink.c:5885:gst_base_sink_change_state:<filesink0> error: Failed to start
2024-06-19T15:08:43.594743Z ERROR kooha::window: Failed to start recording

Caused by:
    0: Failed to initialize pipeline state to playing
    1: Element failed to change its state

Stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: g_main_context_iteration
   6: g_application_run
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: __libc_start_main
  12: <unknown>

I have a Gnome virtual machine (Fedora Silverblue 40) I tested it there and it works as expected. It is on "niri" (Fedora Everything 40) where I doesn't work, also I used flatseal to enable file read and write permissions. Not sure what happens really.

[SeaDve]

The vd folder is probably missing, if that is the case, I'm not sure why it was not created.

[thepragmaticmero]

the directory IS created

$ ls ~/vd
Kooha-2024-06-20-06-56-49.mp4

If we look closely, the Kooha.mp4 file has 0 bytes on it, so no actual recording was made.

$ du ~/vd/Kooha-2024-06-20-06-56-49.mp4
0   /home/senati/vd/Kooha-2024-06-20-06-56-49.mp4

It works fine on Gnome as I said, it's on niri where it has problems, let me test on sway just to be sure.

[thepragmaticmero]

Works on sway.... huh. This is weird.

[thepragmaticmero]

$ sudo journalctl --no-hostname -eb /usr/bin/niri | tail -n 11 | wl-copy

Jun 20 07:21:28 niri[13861]: 2024-06-20T12:21:28.985830Z DEBUG niri::dbus::mutter_screen_cast: record_monitor connector="DP-1" properties=RecordMonitorProperties { cursor_mode: Some(Embedded), _is_recording: None }
Jun 20 07:21:28 niri[13861]: 2024-06-20T12:21:28.987307Z DEBUG niri::dbus::mutter_screen_cast: start
Jun 20 07:21:28 niri[13861]: 2024-06-20T12:21:28.987352Z DEBUG niri::niri: StartCast session_id=3
Jun 20 07:21:28 niri[13861]: 2024-06-20T12:21:28.987403Z DEBUG niri::pw_utils: pw stream: state changed: Unconnected -> Connecting
Jun 20 07:21:28 niri[13861]: 2024-06-20T12:21:28.988927Z DEBUG niri::pw_utils: pw stream: state changed: Connecting -> Paused
Jun 20 07:21:28 niri[13861]: 2024-06-20T12:21:28.988943Z DEBUG niri::pw_utils: pw stream: sending signal with 68
Jun 20 07:21:29 niri[13861]: 2024-06-20T12:21:29.030720Z DEBUG niri::pw_utils: pw stream: state changed: Paused -> Error("no more input formats")
Jun 20 07:21:29 niri[13861]: 2024-06-20T12:21:29.030735Z  WARN niri::pw_utils: pw error id=2 seq=43 res=-32 no more input formats
Jun 20 07:21:29 niri[13861]: 2024-06-20T12:21:29.044509Z DEBUG niri::dbus::mutter_screen_cast: stop
Jun 20 07:21:29 niri[13861]: 2024-06-20T12:21:29.044652Z DEBUG niri::niri: StopCast session_id=3
Jun 20 07:21:29 niri[13861]: 2024-06-20T12:21:29.044809Z DEBUG niri::pw_utils: pw stream: state changed: Paused -> Unconnected

Does the journalctl output tell you something? O-O I'm opening in issue on the niri repo too.

[YaLTeR]

The problem is that Kooha wants shm and niri doesn't implement shm. Arguably, it's better for Kooha to learn to deal with dmabufs, because shm is slow. :) But niri could also learn to provide shm.

[SeaDve]

Hello, thanks for the insights. I'd like to ask if this is still reproducible in the main branch

[YaLTeR]

Is there a nightly flatpak of some sort to try?

[SeaDve]

CI creates a flatpak bundle: https://github.com/SeaDve/Kooha/actions/runs/10033656261/artifacts/1724108935

[YaLTeR]

Hm, still doesn't want to negotiate for some reason:

└─ env RUST_LOG=debug GST_DEBUG=3 flatpak run io.github.seadve.Kooha.Devel
2024-07-22T06:20:17.611839Z  INFO kooha::application: Kooha (io.github.seadve.Kooha.Devel)
2024-07-22T06:20:17.611857Z  INFO kooha::application: Version: 2.3.0-ef6a510 (Devel)
2024-07-22T06:20:17.611863Z  INFO kooha::application: Datadir: /app/share/kooha
2024-07-22T06:20:19.796688Z DEBUG kooha::recording: Created screencast proxy version=Ok(5) available_cursor_modes=Ok(CursorMode(HIDDEN | EMBEDDED | METADATA)) available_source_types=Ok(SourceType(MONITOR | WINDOW | VIRTUAL))
2024-07-22T06:20:19.799733Z DEBUG kooha::recording: Selecting sources cursor_mode=CursorMode(EMBEDDED) source_type=SourceType(MONITOR | WINDOW) is_multiple_sources=true restore_token="7aabbb73-0fca-4855-9525-f39aab08a5d6" persist_mode=Persistent
2024-07-22T06:20:19.807617Z DEBUG kooha::recording: Starting session window_identifier=wayland:JrlofjN03jmSfZ2rxerO3nAtuGA2qlqW
2024-07-22T06:20:19.823009Z DEBUG kooha::pipeline: file_path=/var/home/yalter/Videos/Kooha/Kooha-2024-07-22-09-20-19.webm framerate=Fraction(Ratio { numer: 30, denom: 1 }) profile="webm-vp8" fd=17 stream_len=1 streams=[Stream { node_id: 95, id: Some("0"), position: Some((0, 0)), size: Some((2560, 1440)), source_type: Some(SourceType(MONITOR)) }] record_desktop_audio=true record_microphone=false select_area_data=None
2024-07-22T06:20:19.840201Z DEBUG kooha::device: Finding device name for class `Sink`
2024-07-22T06:20:19.840234Z DEBUG kooha::device: Skipping device `pulsedevice0` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840261Z DEBUG kooha::device: Skipping device `pulsedevice1` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840281Z DEBUG kooha::device: Skipping device `pulsedevice2` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840299Z DEBUG kooha::device: Skipping device `pulsedevice3` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840317Z DEBUG kooha::device: Skipping device `pulsedevice4` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840335Z DEBUG kooha::device: Skipping device `pulsedevice5` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840355Z DEBUG kooha::device: Skipping device `pulsedevice6` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840373Z DEBUG kooha::device: Skipping device `pulsedevice7` as it has unknown device class `Audio/Source`
2024-07-22T06:20:19.840407Z DEBUG kooha::device: Skipping device `pulsedevice8` as it is not the default
2024-07-22T06:20:19.840713Z DEBUG kooha::pipeline: Found desktop audio with name `alsa_output.usb-Generic_ATH-M50xSTS-USB-00.analog-stereo.monitor`
0:00:02.253140768     2 0x7f1408000b90 FIXME                default gstutils.c:4088:gst_element_decorate_stream_id_internal:<pipewiresrc0> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:02.304798083     2 0x7f13f010b150 WARN             pipewiresrc gstpipewiresrc.c:704:on_state_changed:<pipewiresrc0> error: stream error: no more input formats
0:00:02.304862644     2 0x7f1408000b90 WARN                 basesrc gstbasesrc.c:3177:gst_base_src_loop:<pipewiresrc0> error: Internal data stream error.
0:00:02.304879255     2 0x7f1408000b90 WARN                 basesrc gstbasesrc.c:3177:gst_base_src_loop:<pipewiresrc0> error: streaming stopped, reason not-negotiated (-4)
2024-07-22T06:20:19.909347Z ERROR kooha::window: Failed to start recording

Caused by:
    0: Failed to initialize pipeline state to playing
    1: Element failed to change its state

(kooha:2): Gtk-CRITICAL **: 09:20:26.669: Widget of type “AdwAlertDialog” already has an accessible role of type “GTK_ACCESSIBLE_ROLE_GENERIC”

[SeaDve]

videoflip apparently doesn't support dmabufs. Decided to add videoconvert before it instead of removing videoflip for now at the cost of performance. This should hopefully fix it: https://github.com/SeaDve/Kooha/actions/runs/10039530586/artifacts/1725438968

[YaLTeR]

I think some kind of pipeline involving glupload + glvideoflip should support them? Looking around here: https://gitlab.gnome.org/GNOME/gnome-shell/-/blob/9569239ec5e774f59bb973adb34196b984c50e85/js/dbusServices/screencast/screencastService.js#L50

[YaLTeR]

Same problem on that build.

[SeaDve]

I think some kind of pipeline involving glupload + glvideoflip should support them? Looking around here: https://gitlab.gnome.org/GNOME/gnome-shell/-/blob/9569239ec5e774f59bb973adb34196b984c50e85/js/dbusServices/screencast/screencastService.js#L50

That should indeed work, but I prospect that it may not work on all devices.

[SeaDve]

Same problem on that build.

Btw, did the error message remains the same?

[YaLTeR]

Yeah. I suspect that it still might not negotiate DMABUF somehow.

[YaLTeR]

I wonder if the problem might be something else..

[SeaDve]

I'll create a build without videoflip to rule out that branch. Also is there some way to test it without having to install niri?

[YaLTeR]

I'm afraid there isn't, since the other negotiating side is in niri.

Btw there's also this WIP xdp sink element that I couldn't yet get to work either: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1405

Might be something on PW/GST side...

[SeaDve]

Here's the version without videoflip: https://github.com/SeaDve/Kooha/actions/runs/10040587098/artifacts/1725689451

I doubt it will fix the issue given https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1405, but maybe it is worth the try. I also recommend using VA H264 profile which can be changed in the preferences.

[YaLTeR]

Tried, same error, and same error with VA H264. I guess I'll try again when I have new PW according to the xdp src MR.

[thepragmaticmero]

Crash Bandicoot