search

swaywm / sway

i3-compatible Wayland compositor
https://swaywm.org
MIT License
14.42k stars 1.1k forks source link

Unable to start sway on Arch Linux running in VMWare Fusion #5834

Open gpanders opened 3 years ago

gpanders commented 3 years ago

Please fill out the following:

These are the errors I see in the log file:

00:00:00.178 [ERROR] [backend/drm/atomic.c:36] Virtual-1: Atomic commit failed (pageflip): Invalid argument
00:00:00.190 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.190 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:00.190 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.190 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:00.192 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.192 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:00.207 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.207 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:01.196 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:01.196 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:02.200 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:02.200 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:12.225 [ERROR] [backend/drm/atomic.c:36] Virtual-1: Atomic commit failed (modeset): Permission denied

I've tried with and without VMWare's "Accelerate 3D Graphics" option. It doesn't seem to make a difference.

emersion commented 3 years ago

Can you obtain DRM debug logs? See https://github.com/swaywm/wlroots/wiki/DRM-Debugging

gpanders commented 3 years ago

dmesg.log

emersion commented 3 years ago 
[ 2965.440924] [drm:vmw_du_cursor_plane_atomic_check [vmwgfx]] *ERROR* surface not suitable for cursor
[ 2965.440934] [drm:drm_atomic_helper_check_planes [drm_kms_helper]] [PLANE:35:plane-1] atomic driver check failed
[ 2965.440952] [drm:drm_atomic_check_only [drm]] atomic driver check for 000000002c4d38b2 failed: -22
gpanders commented 3 years ago 
$ lsmod | grep drm_kms_helper
drm_kms_helper        266240  1 vmwgfx
cec                    73728  1 drm_kms_helper
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
fb_sys_fops            16384  1 drm_kms_helper
drm                   585728  5 vmwgfx,drm_kms_helper,ttm

I've scoured the package manager for every permutation of mesa, drm, dri, and vmware I can think of and it appears I have all appropriate packages installed.

Also, I don't know if this is relevant/helpful, but I can start sway from an already running Gnome session (i.e. running sway from Gnome Terminal). It opens sway in a separate window but everything seems to be working as expected. It's only when started from the tty that it fails to start. That makes me think it's not a driver problem (though I could be wrong).

hedgepigdaniel commented 3 years ago

WLR_NO_HARDWARE_CURSORS=1 sway is a workaround for me.

emersion commented 3 years ago

It sounds like this check fails in the driver:

    if (surface && !surface->snooper.image) {
        DRM_ERROR("surface not suitable for cursor\n");
        ret = -EINVAL;
    }

The snooper image is created in these conditions:

    if (metadata->scanout &&
        metadata->num_sizes == 1 &&
        metadata->sizes[0].width == 64 &&
        metadata->sizes[0].height == 64 &&
        metadata->format == SVGA3D_A8R8G8B8) {

        srf->snooper.image = kzalloc(64 * 64 * 4, GFP_KERNEL);
        if (!srf->snooper.image) {
            DRM_ERROR("Failed to allocate cursor_image\n");
            ret = -ENOMEM;
            goto out_no_copy;
        }
    } else {
        srf->snooper.image = NULL;
    }

We need to figure out which one of these conditions fail.

Can you provide a drm_info log?

hedgepigdaniel commented 3 years ago

drm_info, in windowed mode: https://gist.github.com/hedgepigdaniel/63c37724779f70a5415efe8237f61bca

hedgepigdaniel commented 3 years ago

The "preferred" mode changes in drm_info when resizing the host window, or when switching to and from full screen modes, or enabling/disabling the use of multiple screens in full screen mode.

However, the modes seen by sway according to swaymsg -t get_outputs don't seem to update consistently. When new displays are added or removed they appear (I think usually with the correct resolution), but resizing an existing host window (guest screen) does not seem to cause sway to update the available modes.

Also when guest use of multiple screens is enabled, the cursor is sometimes not able to travel across the entire area of all the screens.

LMK if any of these deserve separate issues - just posting here in case it's related.

hedgepigdaniel commented 3 years ago

The symptom is different when VMware is set to full screen mode - sway starts and works, but the cursor is not visible as in https://github.com/swaywm/sway/issues/3814

drm_info: https://gist.github.com/hedgepigdaniel/bf4a121cace8d9eb0e0e83393dd71a82

hedgepigdaniel commented 3 years ago

I see many messages like this in my kernel log:

[drm:vmw_du_cursor_plane_atomic_check [vmwgfx]] *ERROR* surface not suitable for cursor
gpanders commented 3 years ago

Thanks @hedgepigdaniel, setting WLR_NO_HARDWARE_CURSORS=1 works for me as well. I had seen that recommended in other issues (such as #3814) but I didn't think to try it since the problems in those issues were different than mine.

hedgepigdaniel commented 3 years ago

The snooper image is created in these conditions:

  if (metadata->scanout &&
      metadata->num_sizes == 1 &&
      metadata->sizes[0].width == 64 &&
      metadata->sizes[0].height == 64 &&
      metadata->format == SVGA3D_A8R8G8B8) {

      srf->snooper.image = kzalloc(64 * 64 * 4, GFP_KERNEL);
      if (!srf->snooper.image) {
          DRM_ERROR("Failed to allocate cursor_image\n");
          ret = -ENOMEM;
          goto out_no_copy;
      }
  } else {
      srf->snooper.image = NULL;
  }

We need to figure out which one of these conditions fail.

Preliminary findings to be confirmed: None of this code runs at all. I put a DRM_ERROR in the else block and didn't see any log messages.

Next step is to put more of them at all the early returns earlier in that function and see of one of them is the culprit.

hedgepigdaniel commented 3 years ago

To save me a bunch of confusion - what code in wlroots would you expect to be calling vmw_surface_define_ioctl?

emersion commented 3 years ago

what code in wlroots would you expect to be calling vmw_surface_define_ioctl?

This function is hooked up to the ioctl VMW_CREATE_SURFACE, which I expect would be called when creating a GBM BO (so as a result of a gbm_bo_create call, which should call the driver-specific function to allocate a buffer in Mesa).

emersion commented 3 years ago

Mesa call, if that helps: https://gitlab.freedesktop.org/mesa/mesa/-/blob/dc3505f87ed69ac843cd4ce7d269b2ab01a32ff7/src/gallium/winsys/svga/drm/vmw_screen_ioctl.c#L187

hedgepigdaniel commented 3 years ago

Well, it turns out that mesa doesn't call vmw_surface_define_ioctl - instead it calls vmw_gb_surface_define_ext_ioctl, and that does not set snooper.image.

The choice between the two ioctls is made here on the basis that have_gb_objects is true.

emersion commented 3 years ago

OK. Can you report a Mesa bug about this?

hedgepigdaniel commented 3 years ago

Sure, https://gitlab.freedesktop.org/mesa/mesa/-/issues/4003

IvoMajic commented 3 years ago

WLR_NO_HARDWARE_CURSORS=1 doesn't help in VMWare Fusion 12.1.0 (17195230) (macOS Big Sur host, fresh Arch Linux guest). I get the following error when starting from tty1:

00:00:00.004 [sway/main.c:349] Unable to retrieve socket path

Same issue when starting from GDM. If started from gnome-terminal sway successfully runs in a separate window.

emersion commented 3 years ago

@IvoMajic Please redirect the output of Sway to a file to get full logs: sway -d >sway.log 2>&1. The console is messed up when Sway switches it to graphics mode.

IvoMajic commented 3 years ago

@emersion full log:

https://gist.github.com/IvoMajic/e09a49d12510e1efa61d1c67dfa3c90b

The result is just a black screen, which can only be exited by switching to other tty or shutdown.

emersion commented 3 years ago

You're also getting this:

00:00:00.221 [ERROR] [backend/drm/atomic.c:36] Virtual-1: Atomic commit failed (pageflip): Invalid argument
lvets commented 3 years ago

Is there anything I can do to help? I'm successfully running Sway 1.5.1 on a Macbook Pro 2019 with Big Sur 11.2.2 in VMware Fusion Pro 12.1.0 in a window (not fullscreen). I am setting WLR_NO_HARDWARE_CURSORS=1.

Debug log from my VM: https://gist.github.com/lvets/944d9d351f814567d472000583822b40

Qix- commented 3 years ago

Having the same issue. My dmesg log shows no errors, and the redirected output from sway is an empty file.

This is using VMWare Workstation Player 16 and a fresh install of Arch.

Using WLR_NO_HARDWARE_CURSORS=1 changed nothing.

Running sway immediately freezes the VM and no additional input works. A forced reboot is required.

emersion commented 3 years ago

the redirected output from sway is an empty file.

You need to redirect stderr.

Qix- commented 3 years ago

@emersion I did. I ran sway -d >sway.log 2>&1 exactly as prescribed in https://github.com/swaywm/sway/issues/5834#issuecomment-751126401.

peterlewis commented 3 years ago

Just popping in with a +1 - I am experiencing the same issue when running sway (albeit in a Void Linux VMware VM).

WLR_NO_HARDWARE_CURSORS=1 did not resolve.

00:00:00.225 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:00.225 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:00.230 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:00.230 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:00.285 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:00.285 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:01.236 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:01.236 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:02.241 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:02.241 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:03.248 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:03.248 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:04.254 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:04.254 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:05.261 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:05.261 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:06.266 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:06.266 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:07.273 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:07.273 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:08.279 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:08.279 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:09.284 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:09.284 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:10.291 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:10.291 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
asininemonkey commented 3 years ago

Sway works perfectly for me when setting WLR_NO_HARDWARE_CURSORS=1 and ensuring Accelerate 3D Graphics in the guest VM's display preferences is enabled.

If the former is not set you will get a black screen. If the latter is not set you will get a Could not connect to socket error.

galaxy4public commented 2 years ago

Something fishy is definitely going on with wlroots and virtualised drm drivers (such as virtio-gpu, vmwgfx, etc.). On all virtual machines (tried Parallels Desktop, VMware Fusion, but above there are reports re: Virtualbox too) any wlroots-based compositor can't produce a hardware cursor (either it is not visible or results in a console lockup). At the same time, Weston happily works with all these drm drivers and enables hardware cursor. JFYI.

ClaCodes commented 1 year ago

I am having the same issue on a fresh install of ubuntu in vmware. Same symptoms (black screen, starting from already running gnome session opens sway correctly in window).

WLR_NO_HARDWARE_CURSORS=1 sway is a workaround for me.

Works for me too. I have to disable the graphical login manager with sudo systemctl set-default multi-user and start it with the workaround from the terminal.

fislysandi commented 1 year ago

hey bros doesnt anybody know how to make WLR_NO_HARDWARE_CURSORS=1 sway this work with sddm? because if i try to input it into the .desktop file it doenst work

pls halp.

blackmirroxx commented 1 year ago

I am having the same issue with arch as a guest in qemu, arch kernel vesions: --6.3.1

JanJenicek commented 10 months ago

+1 not much to add host: Win10 Ent; VMware 16 PRO guest: Debian12 headless; sway 1.7 from scratch + swaybg and all recommended WLR_NO_HARDWARE_CURSORS=1

black screen after sway launched from terminal

after switching between TTYs, graphical output is updated, but again hangs until another TTY switch back and forward.

all env.var. SWAYSOCK etc. appear to be empty by default. Manually setting SWAYSOCK has no effect.

Anything to provide or help?