[SUGGESTION] Mirror DRM/KMS camera preview, across multiple monitors

raspberrypi / picamera2

New libcamera based python library

BSD 2-Clause "Simplified" License

897 stars 190 forks source link

[SUGGESTION] Mirror DRM/KMS camera preview, across multiple monitors #1073

Open AlexHCC opened 4 months ago

AlexHCC commented 4 months ago

Hi, I have a simple python program that shows a camera preview in DRM/KMS mode. When I connect two monitors via the two microHDMI ports on the pi, and start the application, the DRM camera preview only shows on one, and the other one just shows the command line. I wish it was possible to mirror the camera DRM/KMS preview on multiple connected monitors. The functionality could possibly be specified as an argument in the start_preview function, or maybe as a configuration parameter.

Here is a link to a discusion I had about this topic in the rasberry pi forums. Is there any workaround I can use? Any pointers on how I can achieve this currently? Thanks!

davidplowman commented 4 months ago

Hi, I'm afraid I don't know of any workarounds. I guess the 2nd monitor would show up as a second "card" or "connector" (or something, I don't really know anything about DRM/KMS), so there would have to be some work done to implement that. I haven't been planning to do anything in this area myself, mostly because I don't know anything about it, but would be happy to take advice (or PRs!) from anyone who does...

6by9 commented 4 months ago

As I commented on the forum thread, each display will show up as a separate connector, eg HDMI-A-1, HDMI-A-2, DSI-1, etc. On Pi0-4 they'll all be on the same DRM card. PI5 you have DSI, DPI, and composite outputs on separate cards. SPI devices would also be separate cards.

Each connector will connect to a unique crtc. Each crtc will need a unique plane for the image. However you can present the same buffer to multiple planes via the same atomic commit.

Now how to structure that is a trickier proposition.

Each card will need to be handled independently, largely as independent instances of class DrmPreview.
Each connector on a card will need to handle within one instance of class DrmPreview as we want one commit for all displays as they are tied to vsyncs. The instance will need to store the connector, crtc, and plane for each active output.

Actually there is a dual display example in https://github.com/tomba/pykms/blob/master/utils/dual-display.py. If I understood Python import magic better then I might be able to get it to run, but am currently failing.

AlexHCC commented 4 months ago

Actually there is a dual display example in https://github.com/tomba/pykms/blob/master/utils/dual-display.py. If I understood Python import magic better then I might be able to get it to run, but am currently failing.

Unfortunately, I can't get it to run on the pi as I get the following error:

  File "/home/pi/pykms-master/simple-test.py", line 113, in main
    card = kms.Card()
        ^^^^^^^^^^
  File "/home/pi/pykms-master/kms/card.py", line 26, in __init__
    dev_path = Card.__open_first_kms_device()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/pykms-master/kms/card.py", line 53, in __open_first_kms_device
    fcntl.ioctl(fd, kms.uapi.DRM_IOCTL_MODE_GETRESOURCES, res, True)
OSError: [Errno 95] Operation not supported

This happens on any of the utility example scripts I try to run. Do I need to enable something? Or maybe does the pi kms not support this specific action??

6by9 commented 4 months ago

I don't generally do Python (give me C any day!), and there appears to be some overlap with the library being kms vs pykms so we get weird imports around https://github.com/raspberrypi/picamera2/blob/main/picamera2/previews/drm_preview.py#L7-L11

Picamera2's drm preview also uses self.card = pykms.Card();, so there's nothing fundamentally wrong in that. I assume you're not running X or picamera at the same time - the normal thing of DRM only supporting a single master at any point in time still applies.

AlexHCC commented 4 months ago

Now how to structure that is a trickier proposition.

Each card will need to be handled independently, largely as independent instances of class DrmPreview.

Each connector on a card will need to handle within one instance of class DrmPreview as we want one commit for all displays as they are tied to vsyncs. The instance will need to store the connector, crtc, and plane for each active output.

Thanks for the help, I got it working on the pi 4 by creating two unique connectors, crtcs and planes, as described, using a single atomic commit. Currently though its hardcoded to HDMI-A-1, HDMI-A-2, and a single DRM card the pi 4 uses, so it's probably not suitable for a pr.

6by9 commented 4 months ago

Having had a quick chat with those who know a little more about the Python side than I do, it looks to be a bit of a mess with multiple very similar libraries kicking around.

Picamera2 is using the Python bindings from https://github.com/tomba/kmsxx/
The pykms library from the same developer at https://github.com/tomba/pykms is very similar but not the same.
If you use pip to try and install a package called kms, you get some database related package (https://github.com/Py-KMS-Organization/py-kms)

The correct Python package appears to be rpi-kms - https://pypi.org/project/rpi-kms/.

But it sounds like you found a solution without needing the example I'd found in the irrelevant pykms library - congrats! If you felt like throwing it at Github and creating a draft PR (just so it doesn't accidentally get merged), it'd be useful to compare and see if there is a moderately easy way to adopt the general scheme.