Vulkan support?

[johnbchron]

HoneyKrisp is released, so I was wondering what steps need to be taken to port this over here, and what I can do to help.

My knowledge of this is pretty much "we need to update our mesa branch and update some kernel flags or smth, right?" so I don't know much, but I am very willing to help.

[waltmck]

I've been using this fork from @zzywysm, which has had Vulkan support for a couple of weeks. It also includes several noticeable performance improvements which could not be upstreamed, notably:

• compile-time architecture optimizations since we are already building from source
• disabling several unnecessary modules to decrease boot time
• Using the recommended IO scheduler for NVME drives
• Building the Asahi kernel modules into the kernel rather than loading them as modules.

It has been pretty much plug and play after I updated my CPU features list to the M2's. Boot times are decreased and I've noticed longer battery life, so until @tpwrules adds Vulkan support I recommend trying it! It should be clear how to add Honeykrisp to the kernel from looking at that repo.

There is also the separate, and more difficult, issue of how to package Fedora's userspace microvm/FEX/WINE/DXVK stack--I have been struggling to get this to work. Right now only running (a specific version of) Steam is supported and my impression is that it still fairly buggy, so it might make sense to wait until it is in a less-alpha state to consider packaging.

[matteocavestri]

@waltmck How did you packaged MicroVM/FEX/WINE/DXVK? I've looked to the fork you mentioned, but I've not found it. I've also looked at your fork and your config. We can package that stack in this repo using something like ...wineSupport.enable to enable that microVM

[rowanG077]

@matteocavestri Nothing you listed is packaged. Those have nothing to do with vulkan support.

[matteocavestri]

@rowanG077 I know it, but @waltmck spoke about that stack and It could be interesting to adopt here since fedora asahi remix have packaged that.

[waltmck]

@matteocavestri I haven't tried packaging "the right way" with Nix, but should be possible to get working in a Fedora distrobox with their packaging (and I managed to run Steam with FEX and the microvm this way). However, I haven't had time to figure out why mesa drivers are not visible inside of the container (I tried symlinking in several different files from the Nix store but glxinfo -B still fails to load Honeykrisp and falls back to llvmpipe).

[foldfree]

I’ve tried zzywysm’s fork but the Vulkan driver is not loading at all, even on native programs.  Do I need to configure something ? I am on unstable using hyprland without flakes.

[oliverbestmann]

You can also try my fork, vulkan works perfectly (e.g zeditor now runs). Contrary to zzywysm, I have not changed anything from tpwrules repository, except mesa & kernel.

[foldfree]

Thank you Oliver. I was actually using your fork before ;) but had the same issue. I’m on a m2 air. Should I change anything in my config?

  hardware.asahi = {
    withRust = true;
    useExperimentalGPUDriver = true;
    experimentalGPUInstallMode = "replace";
    setupAsahiSound = true;
  };
  hardware.asahi.peripheralFirmwareDirectory = ./firmware;
  boot = {
    consoleLogLevel = 0;
    kernelParams = [ "apple_dcp.show_notch=1" ];
  };

glxinfo -B

``` name of display: :0 display: :0 screen: 0 direct rendering: Yes Extended renderer info (GLX_MESA_query_renderer): Vendor: Mesa (0xffffffff) Device: Apple M2 (G14G B0) (0xffffffff) Version: 24.3.0 Accelerated: yes Video memory: 15669MB Unified memory: yes Preferred profile: core (0x1) Max core profile version: 4.6 Max compat profile version: 4.6 Max GLES1 profile version: 1.1 Max GLES[23] profile version: 3.2 Memory info (GL_ATI_meminfo): VBO free memory - total: 15669 MB, largest block: 15669 MB VBO free aux. memory - total: 0 MB, largest block: 0 MB Texture free memory - total: 15669 MB, largest block: 15669 MB Texture free aux. memory - total: 0 MB, largest block: 0 MB Renderbuffer free memory - total: 15669 MB, largest block: 15669 MB Renderbuffer free aux. memory - total: 0 MB, largest block: 0 MB Memory info (GL_NVX_gpu_memory_info): Dedicated video memory: 15669 MB Total available memory: 15669 MB Currently available dedicated video memory: 15669 MB OpenGL vendor string: Mesa OpenGL renderer string: Apple M2 (G14G B0) OpenGL core profile version string: 4.6 (Core Profile) Mesa 24.3.0-devel OpenGL core profile shading language version string: 4.60 OpenGL core profile context flags: (none) OpenGL core profile profile mask: core profile OpenGL version string: 4.6 (Compatibility Profile) Mesa 24.3.0-devel OpenGL shading language version string: 4.60 OpenGL context flags: (none) OpenGL profile mask: compatibility profile OpenGL ES profile version string: OpenGL ES 3.2 Mesa 24.3.0-devel OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20 ```

[oliverbestmann]

What problem do you have? This looks fine. If you want to try vulkan you can run vkgears -info

[foldfree]

 mpv fail to load the vulkan decoder, zeditor does not open, it’s as if there is no vulkan driver. My EFI stub for asahi is macOS 13.5 but i’m on 15.0.1 on the macOS partition. Could it be the reason? Also does vkgears looks good?

vkgears -info

``` apiVersion = 1.3.292 driverVersion = 6002063 vendorID = 10005 deviceID = 0000 deviceType = integrated GPU deviceName = Apple M2 (G14G B0) deviceExtensions: VK_KHR_8bit_storage VK_KHR_16bit_storage VK_KHR_bind_memory2 VK_KHR_buffer_device_address VK_KHR_copy_commands2 VK_KHR_create_renderpass2 VK_KHR_dedicated_allocation VK_KHR_depth_stencil_resolve VK_KHR_descriptor_update_template VK_KHR_device_group VK_KHR_draw_indirect_count VK_KHR_driver_properties VK_KHR_dynamic_rendering VK_KHR_dynamic_rendering_local_read VK_KHR_external_fence VK_KHR_external_fence_fd VK_KHR_external_memory VK_KHR_external_memory_fd VK_KHR_format_feature_flags2 VK_KHR_get_memory_requirements2 VK_KHR_global_priority VK_KHR_image_format_list VK_KHR_imageless_framebuffer VK_KHR_incremental_present VK_KHR_index_type_uint8 VK_KHR_line_rasterization VK_KHR_load_store_op_none VK_KHR_maintenance1 VK_KHR_maintenance2 VK_KHR_maintenance3 VK_KHR_maintenance4 VK_KHR_maintenance5 VK_KHR_maintenance6 VK_KHR_map_memory2 VK_KHR_multiview VK_KHR_pipeline_executable_properties VK_KHR_pipeline_library VK_KHR_push_descriptor VK_KHR_relaxed_block_layout VK_KHR_sampler_mirror_clamp_to_edge VK_KHR_sampler_ycbcr_conversion VK_KHR_separate_depth_stencil_layouts VK_KHR_shader_draw_parameters VK_KHR_shader_expect_assume VK_KHR_shader_float16_int8 VK_KHR_shader_float_controls VK_KHR_shader_float_controls2 VK_KHR_shader_integer_dot_product VK_KHR_shader_maximal_reconvergence VK_KHR_shader_non_semantic_info VK_KHR_shader_subgroup_extended_types VK_KHR_shader_subgroup_rotate VK_KHR_shader_subgroup_uniform_control_flow VK_KHR_shader_terminate_invocation VK_KHR_spirv_1_4 VK_KHR_storage_buffer_storage_class VK_KHR_swapchain VK_KHR_swapchain_mutable_format VK_KHR_synchronization2 VK_KHR_timeline_semaphore VK_KHR_uniform_buffer_standard_layout VK_KHR_variable_pointers VK_KHR_vertex_attribute_divisor VK_KHR_vulkan_memory_model VK_KHR_workgroup_memory_explicit_layout VK_KHR_zero_initialize_workgroup_memory VK_EXT_4444_formats VK_EXT_attachment_feedback_loop_layout VK_EXT_border_color_swizzle VK_EXT_buffer_device_address VK_EXT_color_write_enable VK_EXT_custom_border_color VK_EXT_depth_clip_enable VK_EXT_descriptor_indexing VK_EXT_dynamic_rendering_unused_attachments VK_EXT_extended_dynamic_state VK_EXT_extended_dynamic_state2 VK_EXT_extended_dynamic_state3 VK_EXT_external_memory_dma_buf VK_EXT_global_priority VK_EXT_global_priority_query VK_EXT_graphics_pipeline_library VK_EXT_host_image_copy VK_EXT_host_query_reset VK_EXT_image_2d_view_of_3d VK_EXT_image_drm_format_modifier VK_EXT_image_robustness VK_EXT_index_type_uint8 VK_EXT_inline_uniform_block VK_EXT_line_rasterization VK_EXT_load_store_op_none VK_EXT_multi_draw VK_EXT_mutable_descriptor_type VK_EXT_non_seamless_cube_map VK_EXT_physical_device_drm VK_EXT_pipeline_creation_cache_control VK_EXT_pipeline_creation_feedback VK_EXT_pipeline_protected_access VK_EXT_pipeline_robustness VK_EXT_primitive_topology_list_restart VK_EXT_private_data VK_EXT_provoking_vertex VK_EXT_robustness2 VK_EXT_sample_locations VK_EXT_scalar_block_layout VK_EXT_separate_stencil_usage VK_EXT_shader_demote_to_helper_invocation VK_EXT_shader_module_identifier VK_EXT_shader_object VK_EXT_shader_replicated_composites VK_EXT_shader_stencil_export VK_EXT_shader_subgroup_ballot VK_EXT_shader_subgroup_vote VK_EXT_shader_viewport_index_layer VK_EXT_subgroup_size_control VK_EXT_swapchain_maintenance1 VK_EXT_texel_buffer_alignment VK_EXT_tooling_info VK_EXT_transform_feedback VK_EXT_vertex_attribute_divisor VK_EXT_vertex_input_dynamic_state VK_GOOGLE_decorate_string VK_GOOGLE_hlsl_functionality1 VK_GOOGLE_user_type VK_VALVE_mutable_descriptor_type update window failed ```

[oliverbestmann]

I can compare once I am home. My efi stub is from a similar time. I assume you're on Wayland, not in xorg? I am using gnome as compositor.

[foldfree]

Thank you for taking the time. I’m using Hyprland v0.44.1-b (Wayland). Could be a bug with Hyprland.

[oliverbestmann]

The vkgears output is exactly the same, except mine is a m1 pro, not a m2. I will push an update to nixpkgs & kernel soonish, maybe that helps.

[oliverbestmann]

Updated the fork to use nixpkgs 41dea55321e5a999b17033296ac05fe8a8b5a257 as well as the most recent kernel.

[foldfree]

thank you i successfully updated, in the meantime I found the issue! https://github.com/hyprwm/Hyprland/issues/8158#issuecomment-2422189625 this is needed for honeykrisp to load in hyprland:

render {
    explicit_sync = 0
}

Thanks for helping, without you i assumed there was something wrong with my nix config and didn’t thought i had to tweak some obscure option in hyprland.

[matteocavestri]

@matteocavestri I haven't tried packaging "the right way" with Nix, but should be possible to get working in a Fedora distrobox with their packaging (and I managed to run Steam with FEX and the microvm this way). However, I haven't had time to figure out why mesa drivers are not visible inside of the container (I tried symlinking in several different files from the Nix store but glxinfo -B still fails to load Honeykrisp and falls back to llvmpipe).

@waltmck To use Vulkan, you need to use the Asahi Mesa drivers within the Fedora container. I've tested this with fedora:40 and fedora:rawhide, and it works without issues. I wrote a bash script because some parameters need to be modified from the default Copr repo to install the multilib x86_64 and i386 drivers:

#!/bin/bash
sudo dnf upgrade --refresh
sudo dnf install 'dnf-command(copr)'

# Asahi Mesa (Optional: Change the filename according to your release)
cat <<EOF | sudo tee /etc/yum.repos.d/group_asahi-mesa-fedora-rawhide.repo > /dev/null
[copr:copr.fedorainfracloud.org:group_asahi:mesa]
name=Copr repo for mesa owned by @asahi
baseurl=https://download.copr.fedorainfracloud.org/results/@asahi/mesa/fedora-\$releasever-\$basearch/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/@asahi/mesa/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1
EOF

# Asahi Mesa x86_64 & i386 (Optional: Change the filename according to your release)
cat <<EOF | sudo tee /etc/yum.repos.d/group_asahi-mesa-fedora-rawhide-x86.repo > /dev/null

[copr:copr.fedorainfracloud.org:group_asahi:mesa:x86_64]
name=Copr repo for mesa owned by @asahi (x86_64)
baseurl=https://download.copr.fedorainfracloud.org/results/@asahi/mesa/fedora-\$releasever-x86_64/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/@asahi/mesa/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1

[copr:copr.fedorainfracloud.org:group_asahi:mesa:ml]
name=Copr repo for mesa owned by @asahi (i386)
baseurl=https://download.copr.fedorainfracloud.org/results/@asahi/mesa/fedora-\$releasever-i386/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/@asahi/mesa/pubkey.gpg
repo_gpgcheck=0
cost=1100
enabled=1
enabled_metadata=1
EOF

# Enable Asahi Steam Repository
sudo dnf copr enable @asahi/steam
sudo dnf upgrade --refresh

# Install Steam
sudo dnf install steam

This setup provides working Mesa drivers and the Asahi-packaged version of Steam. If it helps, I also have benchmark comparisons between vkmark and glmark2 on the host and within Distrobox (macmini 2020 M1 8GB):

	Host	Distrobox
glmark2	4597	3895
vkmark	5773	5429

The only remaining issue is linking libc.so.6 within the microVM and FEX to launch Steam. Maybe you've already managed to address this, as you've previously mentioned. Log for reference:

bin_steam.sh[276]: Setting up Steam content in /home/matteocavestri/Container/distrobox/fedora-steam-test/.local/share/Steam
steam.sh[276]: Running Steam on fedora 40 64-bit
steam.sh[276]: STEAM_RUNTIME is enabled automatically
setup.sh[360]: Updating Steam runtime environment...
 100%   
/home/matteocavestri/Container/distrobox/fedora-steam-test/.local/share/Steam/ubuntu12_32/steam-runtime/run.sh: line 85: steam-runtime-identify-library-abi: command not found
run.sh[3185]: steam-runtime-identify-library-abi --ldconfig-paths failed, falling back to ldconfig
steam.sh[276]: Couldn't find /home/matteocavestri/Container/distrobox/fedora-steam-test/.local/share/Steam/ubuntu12_32/steam-runtime/amd64/usr/bin/srt-logger, logging to console-linux.txt
steam.sh[276]: Error: You are missing the following 32-bit libraries, and Steam may not run:
libc.so.6
Error:
You are missing the following 32-bit libraries, and Steam may not run:
libc.so.6

P.S. I know this discussion focuses on Vulkan, but since it’s somewhat resolved, this thread seems quite relevant to Vulkan support, especially as it’s been promoted by the Asahi community.