Applications like Inochi2D Session want to be able to share a stream to other applications (e.g. OBS Studio) alike to what Spout2 and Syphon allows to do on respectively Windows and macOS with almost zero-latency, so mostly "PipeWire + DMA-BUF" for Linux.
Others applications want to be able to provide a virtual camera (e.g. OBS Studio) to be used as a camera in other applications.
Some of them already does by using v4l2-loopback which require kernel space API to be used.
Also some applications might want to act as virtual microphone (device to send their own audio stream) alike to one of Easy Effects features but without requiring direct access to the host PipeWire.
Idea of a solution
Note for the following:
Shown sequence diagram are not strictly sequence diagram
I didn't deep dive into how PipeWire nodes should be handled, I might do it at the same time that I work on a proof of concept of case n°1
For case n°1
For this use-case, we need to enable applications to provide screen cast sources (virtual) to the ScreenCast portal of the xdp implementation.
So the xdp stack will have to connect to the application D-Bus to make it a screen cast sources provider.
Flatpak already allows the application to use its own app ID as a D-Bus name, so the application could ask xdp to rely on a D-Bus interface implementation of the app.
For edge-case alike Gamescope (Flatpak extension), the user already have to modify the application permission to use it, adding a --own-name should not be seen as an obstacle. Permissions will be set under the application permissions.
Some might point out that D-Bus name can be acquired by only one session of the app, if an application happens to have as a common workflow to have various separated sessions. It's up to the application to workaround that (e.g. add wildcard --own-name=com.example.Application.* to its manifest and suffix the name with a number).
Note: Name in italic are just temporary name
The token sent by the application is meant to allow it to make sure that the caller on its D-Bus is xdg-desktop-portal and not another process.
Deduce the D-Bus path by just converting the name (e.g. com.example.Application -> /com/example/application/), is it a good idea or should we allow the caller propose it's own path as an option ?
It is also imaginable to simplify the per-application implementation by providing a library that provide most of the basic of the implementation.
Since the application provide its own video stream, the application could also provide a audio one paired with the latter.
This could improve the Gamescope sub-usecase, and maybe also allow to some power users of OBS Studio to "stream" an OBS session to another locally without having to rely to solution like NDI or DeckLink if implemented in OBS. Yes, chained OBS Studio setup like that do exist.
For case n°2
I didn't give enough thought about it, and like I said earlier I need to dive a little more on how PipeWire works with Portals.
For this one, we need to enable applications to give a video stream to xdp to make it a camera available through the Camera portal.
There is no expectation that the virtual camera can do more than an actual camera, so format-wise it should be like a real camera.
Remimder: Name in italic are just temporary name
For case n°3
This cases is alike to the case n°2 but relies on Goal: Audio portal/#1129.
But unlike case n°2, the naming "virtual microphone" is just used as a figure of speech.
tytan652
commented
11 months ago
Related to Goal: Peer-to-peer media sharing
Usecases
Case n°1
Applications like Inochi2D Session want to be able to share a stream to other applications (e.g. OBS Studio) alike to what Spout2 and Syphon allows to do on respectively Windows and macOS with almost zero-latency, so mostly "PipeWire + DMA-BUF" for Linux.
We can also point out Gamescope that is a Wayland compositor that want to enable sharing in nested scenarios where a xdp implementation is not a solution (https://github.com/obsproject/obs-studio/pull/9607).
Case n°2
Others applications want to be able to provide a virtual camera (e.g. OBS Studio) to be used as a camera in other applications. Some of them already does by using v4l2-loopback which require kernel space API to be used.
As an example, OBS Studio plans to move features to PipeWire (https://github.com/obsproject/obs-studio/discussions/7998), but the actual proposed implementation requires an access to the host PipeWire.
Case n°3
Also some applications might want to act as virtual microphone (device to send their own audio stream) alike to one of Easy Effects features but without requiring direct access to the host PipeWire.
Idea of a solution
Note for the following:
For case n°1
For this use-case, we need to enable applications to provide screen cast sources (virtual) to the ScreenCast portal of the xdp implementation. So the xdp stack will have to connect to the application D-Bus to make it a screen cast sources provider.
Flatpak already allows the application to use its own app ID as a D-Bus name, so the application could ask xdp to rely on a D-Bus interface implementation of the app.
For edge-case alike Gamescope (Flatpak extension), the user already have to modify the application permission to use it, adding a
--own-nameshould not be seen as an obstacle. Permissions will be set under the application permissions.Some might point out that D-Bus name can be acquired by only one session of the app, if an application happens to have as a common workflow to have various separated sessions. It's up to the application to workaround that (e.g. add wildcard
--own-name=com.example.Application.*to its manifest and suffix the name with a number).Note: Name in italic are just temporary name
The token sent by the application is meant to allow it to make sure that the caller on its D-Bus is xdg-desktop-portal and not another process.
Deduce the D-Bus path by just converting the name (e.g.
com.example.Application->/com/example/application/), is it a good idea or should we allow the caller propose it's own path as an option ?It is also imaginable to simplify the per-application implementation by providing a library that provide most of the basic of the implementation.
Thought about audio on those Screencast
Also related to Goal: Window screencast with audio.
Since the application provide its own video stream, the application could also provide a audio one paired with the latter.
This could improve the Gamescope sub-usecase, and maybe also allow to some power users of OBS Studio to "stream" an OBS session to another locally without having to rely to solution like NDI or DeckLink if implemented in OBS. Yes, chained OBS Studio setup like that do exist.
For case n°2
I didn't give enough thought about it, and like I said earlier I need to dive a little more on how PipeWire works with Portals.
For this one, we need to enable applications to give a video stream to xdp to make it a camera available through the Camera portal.
There is no expectation that the virtual camera can do more than an actual camera, so format-wise it should be like a real camera.
Remimder: Name in italic are just temporary name
For case n°3
This cases is alike to the case n°2 but relies on Goal: Audio portal/#1129.
But unlike case n°2, the naming "virtual microphone" is just used as a figure of speech.