Some notes for UTM/QEMU

vancluever commented 11 months ago

Been working with trying to get bootstrapped off of this setup, thanks @mitchellh!

Couple of things I've noticed as I've been messing with this on UTM:

We need the userspace spice-vdagent (not vdagentd) running in the user's X session, e.g. launched out of .xprofile.
UTM prior to 4.4.0 does not handle accessibility resizing properly which might drive you wild trying to figure out what is going on when troubleshooting dynamic resolution while doing things like snapping to edges in Rectangle.
LIBGL_ALWAYS_SOFTWARE may be unnecessary now, unless I haven't stumbled on issues yet. 😉 I was building from scratch and didn't include this option, and things seemed to be working okay.

I'll add a PR once I get more settled in with my config!

I might also take a stab at adding some automation for dynamic resolution too so that I don't necessarily need to run xrandr-auto super quick after I resize. I noticed the note about udev rules, I'll see if I can find a way to get this tidy so that it's localized to the UTM setup!

mitchellh commented 11 months ago

All sounds great to me. Thanks @vancluever!

vancluever commented 11 months ago

Adding a note on the LIBGL_ALWAYS_SOFTWARE env var - looks like this is needed due to https://github.com/NixOS/nixpkgs/issues/239977 and similar issues. There seem to be a lot of these... apparently there are problems trying to find the right OpenGL driver to use? Continuing to look into this though... I'm seeing the recommendation to use https://github.com/guibou/nixGL, but that seems to be for non-nix host targets.

vancluever commented 11 months ago

So I think this is the core of the issue with UTM: https://github.com/utmapp/UTM/issues/4285. It seems like the bottleneck here is ANGLE limiting the OpenGL implementations actually available to the guest. As a result, the available versions of OpenGL given to a guest are rather old.

There likely is no way around this on QEMU until this is corrected.

Incidentally, using the native Apple hypervisor framework on UTM works just fine using the same NixOS flake. 🙂 (I've built myself a minimal flake in the meantime to work through some of these issues).

vancluever commented 11 months ago

Just a comparison of glxinfo on both VMs:

QEMU

Apple Virtualization

vancluever commented 11 months ago

Looks like this is due to SW rendering on the Apple virtualization side - so yeah, likely software is our only hope for now.

tobiabocchi commented 5 months ago

Hello!! Any updates on this? How is the support for NixOS using UTM on apple silicon?

vancluever commented 5 months ago

@tobiabocchi I've moved on to my own config and VMware Fusion, but from what I understand, the situation with OpenGL and QEMU is still not much better. The aforementioned UTM issue still seems to be open, so that is probably telling of the current status.

That said UTM works just fine on Apple Silicon in general and software graphics, and the same goes for NixOS.

fspillner commented 5 months ago

@vancluever @mitchellh This is off-topic.

I don't get the 3d acceleration working on my VMware Fusion 13.5.1 w/ NixOS in my web browsers e.g. Firefox or Chromium.

With NixOS 23.11 or unstable, Accelerated is set to no.

$ glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: VMware, Inc. (0x15ad)
    Device: SVGA3D; build: RELEASE;  LLVM; (0x406)
    Version: 24.0.2
    Accelerated: no
    Video memory: 1MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.3
    Max compat profile version: 4.3
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: VMware, Inc.
OpenGL renderer string: SVGA3D; build: RELEASE;  LLVM;
OpenGL core profile version string: 4.3 (Core Profile) Mesa 24.0.2
OpenGL core profile shading language version string: 4.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.3 (Compatibility Profile) Mesa 24.0.2
OpenGL shading language version string: 4.30
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 24.0.2
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

But the benchmarking with glmark2 shows good performance. How do you manage to get it work?

vancluever commented 5 months ago

@fspillner yeah interesting, honestly I never noticed that, but I have that as well.

Are you on aarch64? Unfortunately I don't have an Intel Mac to test on to eliminate.

I don't think there's an easy answer - there doesn't seem to be many issues for it in nixpkgs and the ones that are open are inconclusive, see https://github.com/NixOS/nixpkgs/issues/239598 - not exactly related, but yeah.

I guess like you, performance has been good so I haven't needed to mess with it too much, heh. FWIW, my daily NixOS driver these days is bare metal so that probably helps as well.

mitchellh / nixos-config

Some notes for UTM/QEMU #37