search

swaywm / wlroots

A modular Wayland compositor library
https://gitlab.freedesktop.org/wlroots/wlroots/
MIT License
2.15k stars 341 forks source link

Displaylink support #1823

Open Algram opened 5 years ago

Algram commented 5 years ago

Hello,

I wanted to ask if displaylink is something that this project wants to support in the future. I've composed a list of resources to make it easier to work on this issue:

Why support displaylink?

We all know that the protocol itself is probably going away at some point in the future. Right now though, there are a lot of companies that have docking stations that ONLY work with displaylink. Supporting the protocol will allow wlroots users access to those adapters.

Current problems (afaik)

How to

There is the evdi (https://github.com/DisplayLink/evdi) kernel module that already works ok. Every wayland compositor needs to implement it though. Mutter has done that already (https://github.com/DisplayLink/evdi/issues/58#issuecomment-495247935), so maybe looking at their implementation helps moving this forward in wlroots.

wlroots has migrated to gitlab.freedesktop.org. This issue has been moved to:

https://gitlab.freedesktop.org/wlroots/wlroots/-/issues/1823

ascent12 commented 5 years ago

Yeah, this is something I eventually want to support. I even bought a displaylink device a while ago for this purpose. It has been a low priority while some rather drastic backend changes are being made, which should eventually lead to us being able to support devices without their own rendering capabilities, like displaylink ones.

Another comment I made a while ago related to this: https://github.com/swaywm/wlroots/issues/1278#issuecomment-492514451

Algram commented 4 years ago

Alright thanks for your answer! I think I already know the answer, but let me ask anyway: Is there such a thing as a quick and dirty workaround to get at least display output working?

Otherwise I will unfortunately have to switch back to bspwm since my workplace only provides displaylink stations.

Thanks a ton!

dstoe commented 4 years ago

I came here since I tried to get started with sway and my display-link port-replicator. Update from my today's debugging session:

pati-ni commented 3 years ago

I may have a Displaylink device to spare. Is this still under development? Will this be of any use?

emersion commented 3 years ago

It depends which generation of displaylink this is. The more recent generations have a proprietary driver.

pati-ni commented 3 years ago

It is an HP USB-C Universal Dock. Currently, it only works after I spin up gnome 3 with evdi 1.7 module and the proprietary Displaylink v5.3.1.34-4.

Not sure about the protocol version though. I found no way to flush any firmware to this thing which is odd.

emersion commented 3 years ago

Ah, so probably it's one of the newer ones. I'm not really interested in helping with that, but feel free to send pull requests.

progandy commented 3 years ago

There are some options to use/debug the evdi driver without displaylink. Most of the non-displaylink use cases can be covered with compositor support like create_output in sway and wf-recorder / wayvnc, though.

https://github.com/xytovl/evdi-hello https://github.com/dgarfias/screx

ascent12 commented 3 years ago

Even if evdi is "open source", it effectively only exists to push a proprietary driver. Any kind of special support for it should not be in wlroots.

dstoe commented 3 years ago

Spent some time digging into this. I'm not sure to how far this is an evdi issue. The mentioned issue "evdi_dri.so not present" is totally fine, since evdi is an output-only device with no rendering capabilities.

My observations

Open Questions

emersion commented 3 years ago

How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?

It's a little tricky. Maybe the kernel solution described here can help, but maybe not: https://lists.freedesktop.org/archives/dri-devel/2021-April/303692.html

As of today, all we're left with are heuristics.

We can discover whether a DRM device has rendering capabilities by checking if it has a render node. If it doesn't have a render node, there's a trap: maybe a completely different DRM device can be used to render to buffers allocated on the first device (split render/display SoC).

We can try directly displaying a buffer coming from a different device with KMS. We need to perform an atomic test-only commit to make sure that works. We already support this for client buffers (see drm_connector_test). We could support it for buffers coming from the primary device.

Maybe this will work, but maybe this will be sub-optimal. We can also try to do a CPU copy of the buffer. We don't quite yet have the architecture for this, I suspect we'll need something like DRM dumb buffers. Ref https://github.com/swaywm/wlroots/pull/2700.

Also ref https://github.com/swaywm/wlroots/issues/1347, which is about improving the whole multi-GPU copy situation.

Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

Yeah, it should work. But from what you wrote it seems like this is just a llvmpipe bug when multi-threading is enabled? And it seems like it works fine when disabling multi-threading?

pati-ni commented 2 years ago

Spent some time digging into this. I'm not sure to how far this is an evdi issue. The mentioned issue "evdi_dri.so not present" is totally fine, since evdi is an output-only device with no rendering capabilities.

My observations

  • First the good story: sway starts perfectly fine if the copy from the primary to the secondary drm device was disabled (writing this message from sway using the displaylink station heart_eyes ). Instead, simply re-interpret the buffer from the rendering-device as if it was usable on the output device, too. Hope you forgive this dirty unsafe hacking for testing see_no_evil This works with my i915 rendering node which also uses the CPU-RAM. I hacked this into wlroots/backend/drm/renderer.c::drm_plane_lock_surface:
  // [...]
  struct wlr_buffer *local_buf;
  // Change the following check to always use the `else`-path even if a parent-device is used.
  if (drm->parent && 0) {
      // Perform a copy across GPUs
      local_buf = drm_surface_blit(&plane->mgpu_surf, buf);
      if (!local_buf) {
          wlr_log(WLR_ERROR, "Failed to blit buffer across GPUs");
          return false;
      }
  } else {
      local_buf = wlr_buffer_lock(buf);
  }
  • The copy-path also works if the LP_NUM_THREADS was set to 1 forcing the software-rasterizer from mesa to work single-threaded. Then, however, the rendering is unusable slow.
  • With a multi-threaded software-rasterizer, there seems to be a race-condition which makes the llvmpipe component of mesa crash with a bus-error (see referenced issue in my comment from May 29th).

Open Questions

  • How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?
  • Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

Unfortuantely, @dstoe 's hack did not work for me. At the time of writing the code modified to get displaylink up and running has been moved to backend/drm/drm.c::drm_connector_set_pending_fb

@dstoe Can you give us a bit information about the model of your displaylink docking station and/or setup you tried it with? I can say that not all usb type C docking stations are created equally. Sway works out of the box with a Lenovo thinkpad type-C docking station while it fails with a Dell one. I am suspecting that the Lenovo thinkpad one is doing displayport over the type-c usb but I am not sure on this one.

spaceone commented 2 years ago

This hit me, too. For google searchability here are the symtoms:

swaynag: swaynag/swaynag.c:390: swaynag_setup: Assertion `swaynag->compositor && swaynag->layer_shell && swaynag->shm' failed.

Aborted (core dumped)
dstoe commented 2 years ago

@dstoe Can you give us a bit information about the model of your displaylink docking station and/or setup you tried it with?

My tests were with a Fujitsu port replicator pr8.1. I didn't find any details on which displaylink chip version is included. The manual only states: "Displaylink Certified".

I can say that not all usb type C docking stations are created equally. Sway works out of the box with a Lenovo thinkpad type-C docking station while it fails with a Dell one. I am suspecting that the Lenovo thinkpad one is doing displayport over the type-c usb but I am not sure on this one.

It looks to me as if the Lenovo replicator is not using the DisplayLink technology so maybe the video-signal is directly transmitted over usb-c where the scan-out (conversion to an appropriate video signal) happens in the laptop-graphics card as usual. In the DisplayLink technology only the rendering to the frame-buffer happens on the laptop-graphics card. The frame-buffer is then transmitted to the replicator so it's the replicators duty to do the scan-out. This splitting of the rendering and scan-out to different devices is here the involved part why it doesn't work out of the box (although it should). The hack I posted only works if the laptop has a graphics-card which does the rendering to the CPU-RAM. With the hack the source frame-buffer is assumed to be readable by the output-device driver instead of performing an appropriate cross-device copy (which would e.g. be necessary if it was a discrete graphics card). It's also relying on the dangerous hope that the source frame-buffer would not be freed until the DisplayLink-driver would have transmitted it to the replicator.

All of this is my very rough knowledge of what I learned during debugging and reading on the issue. Very likely that I'm not using the correct terms or that I'm wrong.