search

firatkiral / pypeek

Peek screen recorder and screenshot with annotations
GNU General Public License v3.0
209 stars 9 forks source link

Support Wayland display server for Linux users #6

Open mcmxcdev opened 1 year ago

mcmxcdev commented 1 year ago

Looking at the requirements, they state that only Xorg is supported currently: https://github.com/firatkiral/pypeek#requirements That is unfortunate, since various Linux distros move from Xorg to Wayland as default, so it would make sense to support it if a broad user base is desired.

I was able to start the application and tried to record on Wayland, but after stopping the recording there are no videos rendered, and I see various error stack traces in the CLI output.

firatkiral commented 1 year ago

@mcmxcdev,

Unfortunately, its a wayland security update / choice and it doesn't allow apps to record screen out of the box. It's hard to implement permissions and no enough documentations & tutorials for now.

mcmxcdev commented 1 year ago

Ok, I understand.

It did work in peek and the way I understood it how it was done in peek is by using XWayland: https://wayland.freedesktop.org/xserver.html

So that could be a solution as long as Wayland is so strict about screen recording.

zocker-160 commented 1 year ago

It did work in peek and the way I understood it how it was done in peek is by using XWayland:

IIRC that is not a solution as you can only record applications which are also running in XWayland and any other window will stay black.

I could be wrong, but last time I tried that, it was that way.

probonopd commented 1 year ago

My conclusion is that Wayland is not suitable for me because it breaks too much of what I need. So while it is sad to see that peek is no more, I am very happy that a Qt-based tool to record X11/Xorg now exists. Thank you very much!

That is unfortunate, since various Linux distros move from Xorg to Wayland as default,

Imho, it is unfortunate that "various Linux distros" are following Fedora's lead to push the unfinished, incompatible Wayland onto unsuspecting users. I don't use such distros anymore.

mcmxcdev commented 1 year ago

@probonopd I will be reading through the gist you posted at some point, I think I actually stumbled upon it before and bookmarked it.

Both Manjaro Linux and Ubuntu which I am using are adopting Wayland, so not much I can do about their decision but to ask for support in 3rd party tooling.

marcolaux commented 1 year ago

Wayland makes use of the XDG Portals standard to get the permission:

https://flatpak.github.io/xdg-desktop-portal/

marcolaux commented 1 year ago

My conclusion is that Wayland is not suitable for me because it breaks too much of what I need. So while it is sad to see that peek is no more, I am very happy that a Qt-based tool to record X11/Xorg now exists. Thank you very much!

That is unfortunate, since various Linux distros move from Xorg to Wayland as default,

Imho, it is unfortunate that "various Linux distros" are following Fedora's lead to push the unfinished, incompatible Wayland onto unsuspecting users. I don't use such distros anymore.

It does not just break things. It adds security concepts that the legacy X11 is lacking.

Without pushing new technology the transition will never get done.

It is perfectly possible to implement such functionality. Please educate yourself.

marcolaux commented 1 year ago

Here is a Python example, perhaps it is useful:

https://gitlab.gnome.org/-/snippets/19

Here is an example for screenshots and Portals in Python:

https://stackoverflow.com/a/56384206

I haven't tested it and don't know much about Python though.

zocker-160 commented 1 year ago

It is perfectly possible to implement such functionality. Please educate yourself. I haven't tested it

I think it would help if you would actually know what you are talking about. I have worked a lot with XDG portals in the past and I can assure you that whatever pypeek does currently is completely impossible to implement in Wayland.

The XDG screenshot portal does only screenshot the entire screen or a specific window, that is it.

Pypeek however uses the position of the window + its size to screenshot a specific part of the screen. Unfortunately there is no way to find out the position of the current window because Wayland does not have a coordinate system at all.

It is literally impossible to make a screenshot based on the position of the window because of that.

marcolaux commented 1 year ago

I was more referring to examples that go in that direction. The portal to use would be the screencast, not the screenshot API.

Have a look at the app Kooha, it's written in Rust and supports Wayland to capture just portions of the screen (essentially what this app here does).

zocker-160 commented 1 year ago

The portal to use would be the screencast, not the screenshot API.

Does not change the fact that Wayland doesn't have a coordinate system, which makes pypeek impossible to implement.

Have a look at the app Kooha, it's written in Rust and supports Wayland to capture just portions of the screen (essentially what this app here does).

Not comparable. It works around the limitation of Wayland I mentioned above by always capturing the entire screen and then cutting out the area that the user specified using a selection box.

Again as I already said, pypeek uses the position of the window to get the area to record, but there is no such thing as a window position in Wayland.

marcolaux commented 1 year ago

green-recorder did something similar while using the GNOME screen recording API. It created a window where you can choose the area, like pypeek does. green-recorder closes the window then though and starts recording. But what would you stop from not closing the window and make the experience seamless? It supported Wayland.

zocker-160 commented 1 year ago

Why is this so hard for you to understand that it is not possible?

I took a look into the (sadly pretty horrible) code of pypeek to find out how it works and this call here is the key problem and is not possible on Wayland:

https://github.com/firatkiral/pypeek/blob/4e0fce8b0703af84476ffbdd90f4dd8084b48a93/src/pypeek/main.py#L819

QT Bug report about this issue, closed as "Out of scope" with

[...] this is not possible with current Wayland protocol and shell extensions. Client surfaces are not supposed to know their position on the screen.

Ironically pypeek uses pretty much exactly the same method of capturing the screen area than Peek does and Peek got discontinued because of exactly this method not working on Wayland.

And even if eventually a Wayland protocol might allow this functionality (which I am very sure never will as this is a limitation "by design"), it would require a rewrite of 80% of pypeek's code.

I think this is a lost case.

EDIT: @marcolaux if you are fine with a box selector like 99% of other screen recorders use, then why not use those 99% of other applications?

It created a window where you can choose the area, like pypeek does

No after looking at the code, it is absolutely not "like pypeek does" at all. The box selector in green-recorder comes from the portal and is not something the application is drawing.

The Portal makes you select an area of the screen and then returns a URI to the image file of the screenshot.

marcolaux commented 1 year ago

EDIT: @marcolaux if you are fine with a box selector like 99% of other screen recorders use, then why not use those 99% of other applications?

In fact I do. I just wanted to contribute just in case the maintainer did not look into the examples I made (that might not be applicable, but okay - didn't know - nothing lost, right?).

The box selector in green-recorder comes from the portal and is not something the application is drawing

wouldn't it be possible to create that portal in the client window and communicate via the portal API with it?

zocker-160 commented 1 year ago

In fact I do. I just wanted to contribute just in case the maintainer did not look into the examples I made

@marcolaux Ah ok sorry, I misunderstood you. Ignore what I said then.

wouldn't it be possible to create that portal in the client window and communicate via the portal API with it?

Sadly this is not how Portals work. Let me explain:

If you look at the documentation of the screencast and screenshot Portals, you will see that all you can do is request a screenshot / screencast with a few options like the window name to capture, with and without the mouse etc and then you get either a URI to the screenshot or a handle to a pipewire session back.

The implementation of those Portals is dependent on the desktop environment used (see KDE implementation and GNOME implementation).

Those implementations are the ones which create the window where you can select the area to screenshot, you have no control over that.

The Screencast portal does actually have an option where you can pass coordinates and a size to record, but the fundamental issue is, that on Wayland there is no way for an application to find out where it is located and this is by design.

There are of course hacks to bypass it, e.g. by creating a transparent fullscreen window, but that hack in particular is prone to not working correctly at all on tiled window managers for example.

Also see this issue on the Wayland issue tracker:

https://gitlab.freedesktop.org/wayland/wayland/-/issues/264

Answer from Wayland dev:

No, there is no way to set the absolute position of a window.

AlexDaniel commented 2 months ago

The Screencast portal does actually have an option where you can pass coordinates and a size to record, but the fundamental issue is, that on Wayland there is no way for an application to find out where it is located and this is by design.

But what is so difficult in finding the position of the given window, considering that you have the entire screen stream?

Here are the steps to implement this functionality as I see it. Please let me know why this wouldn't work.

  1. In pypeek itself, add a recognizable border around the window (technically inside the window, but you get the point).
  2. Via xdg portals, request the stream of the entire screen
  3. In the received stream, find the position of the border 4a. Now use those coordinates to cut out the requested part of the screen 4b. Or, optionally, re-request a portal to those specific coordinates, if that is possible
zocker-160 commented 2 months ago

Recording the entire screen and then cropping the area out would work, but live cropping the video does not sound like a trivial task to me (maybe gstreamer has a module for cropping?)

As for 4b, I just checked the latest (v5) of the ScreenCast portal and there you can only select between "Monitor","Window" and "Virutal" (like virtual camera created with v4l2 for example).

So with the current iteration it is not possible to request coordinates.