macOS - Pixel Format issues with MoltenVK

[Veemyu]

There are a few games that, fail at using unsupported pixel formats. Hardware: M1 Pro equipped MacBook Pro 14 Inch Source: Freshly built CEMU from master. I'm just reporting on these issues to gain some attention for them, so people who are smarter than me might have an idea or an easy fix. When I followed Yuzu in it's beginning there were a lot of Pixel Formats added to support a wide range of games. These need to then get swizzled, or something.

The Legend of Zelda: Breath of the Wild crashes because of unsupported pixel formats.

[mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: vkCmdCopyBufferToImage(): The image is using Metal format MTLPixelFormatInvalid as a substitute for Vulkan format VK_FORMAT_R5G6B5_UNORM_PACK16. Since the pixel size is different, content for the image cannot be copied to or from a buffer. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. libc++abi: terminating with uncaught exception of type std::runtime_error: failed to submit command buffer. Error -11 [1] 41390 abort ./Cemu_release

Mario Kart 8 is just as interesting, as it tries to substitute for another pixel format but fails at copying.

[mvk-info] Created 3 swapchain images with initial size (1280, 720) and contents scale 1.0 for screen Built-in Retina Display. [mvk-info] Created 3 swapchain images with initial size (1280, 720) and contents scale 1.0 for screen Built-in Retina Display. validateWithDevice:3718: failed assertionRender Pipeline Descriptor Validation Blending is enabled for render target 0; however, the pixelformat MTLPixelFormatRGBA32Uint for this render target is not blendable. ' [1] 41437 abort ./Cemu_release `

And this is Super Mario 3D World

[mvk-info] Created 3 swapchain images with initial size (1280, 720) and contents scale 1.0 for screen Built-in Retina Display. [mvk-info] Created 3 swapchain images with initial size (1280, 720) and contents scale 1.0 for screen Built-in Retina Display. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: vkCmdCopyBufferToImage(): The image is using Metal format MTLPixelFormatInvalid as a substitute for Vulkan format VK_FORMAT_R5G6B5_UNORM_PACK16. Since the pixel size is different, content for the image cannot be copied to or from a buffer. [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_R5G6B5_UNORM_PACK16 is not supported on this device. libc++abi: terminating with uncaught exception of type std::runtime_error: failed to submit command buffer. Error -11 [1] 41473 abort ./Cemu_release

[UltraHDR]

I commented out the "failed to submit command buffer" code and it doesn't crash anymore. See: https://github.com/cemu-project/Cemu/pull/280

It's a hack for now

[Veemyu]

That's a pretty cool hack for now though :P Works pretty well thank you @UltraHDR It will be interesting to get these pixel formats working for real though :)

[rcaridade145]

Looking at https://github.com/KhronosGroup/MoltenVK/issues/533#issuecomment-484981992 it seems Metal does not support that on desktop.

Edit : https://developer.apple.com/documentation/metal/mtlpixelformat/b5g6r5unorm It seems available on MacOS > 11

[jcrm1]

Looks like it's supported on MoltenVK, too? Weird: link

[rcaridade145]

That means that MoltenVK is not being built correctly https://github.com/KhronosGroup/MoltenVK/blob/0f722b84fb4fb61a8a3c7bcf2168d4fc35d64f43/Common/MVKCommonEnvironment.h#L86 .

It seems that only if the target CPU is ARM it enables that flag.

RPCS3 takes a different approach

https://github.com/RPCS3/rpcs3/blob/2d5d5746d1578db95c10229df3e81f33d7f7a765/rpcs3/Emu/RSX/VK/VKFormats.cpp#L209 https://github.com/RPCS3/rpcs3/blob/d686b48f6549c736661e14d1e0990b043c32e3c2/rpcs3/Emu/RSX/VK/VKGSRender.cpp#L37

[Veemyu]

That's a step closer to a solution. Any way to implement it?

[Tillsunset]

I swapped the pixel format VK_FORMAT_R5G6B5_UNORM_PACK16 with VK_FORMAT_R8G8B8A8_UNORM, now it treats the problematic pixel format with the known good format. Now it loads into Super Mario 3D World, but has [mvk-error] VK_ERROR_FORMAT_NOT_SUPPORTED: VkFormat VK_FORMAT_A1R5G5B5_UNORM_PACK16 is not supported on this device. when loading a save file. As far as I could see all the textures look uncorrupted, so I have high confidence that this would solve many pixel format issues.

As for if Metal and/or Molten-VK support the specific pixel format, it depends on which gpu you are using. Apple has a feature support chart that show which gpus support which features of metal (pixel format). GPU feature list, pixel format support pages 8-11

Another issue encountered was that if you have a combination of demanding game and unoptimized build of Cemu. There's a high chance you encounter a deadlock when loading

[rcaridade145]

If you don't build MoltenVK with brew but for instance using something like this script - https://github.com/ares-emulator/ares/blob/master/thirdparty/MoltenVK/build-moltenvk.sh - from the emulator Ares is the output the same?

[nalz0]

Using the latest complied binary for macOS, I can say the issue for some games has been fixed such as Super Mario 3D World on macOS 13. However, the game : Mario Kart 8 has still got issues. I have not tried BOTW yet.

No bundle id found [mvk-info] MoltenVK version 1.1.11, supporting Vulkan version 1.1.224. The following 81 Vulkan extensions are supported: VK_KHR_16bit_storage v1 VK_KHR_8bit_storage v1 VK_KHR_bind_memory2 v1 VK_KHR_buffer_device_address v1 VK_KHR_create_renderpass2 v1 VK_KHR_dedicated_allocation v3 VK_KHR_depth_stencil_resolve v1 VK_KHR_descriptor_update_template v1 VK_KHR_device_group v4 VK_KHR_device_group_creation v1 VK_KHR_driver_properties v1 VK_KHR_dynamic_rendering v1 VK_KHR_external_fence v1 VK_KHR_external_fence_capabilities v1 VK_KHR_external_memory v1 VK_KHR_external_memory_capabilities v1 VK_KHR_external_semaphore v1 VK_KHR_external_semaphore_capabilities v1 VK_KHR_fragment_shader_barycentric v1 VK_KHR_get_memory_requirements2 v1 VK_KHR_get_physical_device_properties2 v2 VK_KHR_get_surface_capabilities2 v1 VK_KHR_imageless_framebuffer v1 VK_KHR_image_format_list v1 VK_KHR_maintenance1 v2 VK_KHR_maintenance2 v1 VK_KHR_maintenance3 v1 VK_KHR_multiview v1 VK_KHR_portability_subset v1 VK_KHR_push_descriptor v2 VK_KHR_relaxed_block_layout v1 VK_KHR_sampler_mirror_clamp_to_edge v3 VK_KHR_sampler_ycbcr_conversion v14 VK_KHR_separate_depth_stencil_layouts v1 VK_KHR_shader_draw_parameters v1 VK_KHR_shader_float16_int8 v1 VK_KHR_shader_subgroup_extended_types v1 VK_KHR_storage_buffer_storage_class v1 VK_KHR_surface v25 VK_KHR_swapchain v70 VK_KHR_swapchain_mutable_format v1 VK_KHR_timeline_semaphore v2 VK_KHR_uniform_buffer_standard_layout v1 VK_KHR_variable_pointers v1 VK_EXT_buffer_device_address v2 VK_EXT_debug_marker v4 VK_EXT_debug_report v10 VK_EXT_debug_utils v2 VK_EXT_descriptor_indexing v2 VK_EXT_fragment_shader_interlock v1 VK_EXT_hdr_metadata v2 VK_EXT_host_query_reset v1 VK_EXT_image_robustness v1 VK_EXT_inline_uniform_block v1 VK_EXT_memory_budget v1 VK_EXT_metal_objects v1 VK_EXT_metal_surface v1 VK_EXT_post_depth_coverage v1 VK_EXT_private_data v1 VK_EXT_robustness2 v1 VK_EXT_sample_locations v1 VK_EXT_scalar_block_layout v1 VK_EXT_separate_stencil_usage v1 VK_EXT_shader_stencil_export v1 VK_EXT_shader_viewport_index_layer v1 VK_EXT_subgroup_size_control v2 VK_EXT_swapchain_colorspace v4 VK_EXT_texel_buffer_alignment v1 VK_EXT_texture_compression_astc_hdr v1 VK_EXT_vertex_attribute_divisor v3 VK_AMD_gpu_shader_half_float v2 VK_AMD_negative_viewport_height v1 VK_AMD_shader_image_load_store_lod v1 VK_AMD_shader_trinary_minmax v1 VK_IMG_format_pvrtc v1 VK_INTEL_shader_integer_functions2 v1 VK_GOOGLE_display_timing v1 VK_MVK_macos_surface v3 VK_MVK_moltenvk v35 VK_NV_fragment_shader_barycentric v1 VK_NV_glsl_shader v1 [mvk-info] GPU device: model: Apple M1 type: Integrated vendorID: 0x106b deviceID: 0xd0003ef pipelineCacheUUID: 0000277F-0D00-03EF-0000-000000000000 supports the following Metal Versions, GPU's and Feature Sets: Metal Shading Language 2.4 GPU Family Apple 7 GPU Family Apple 6 GPU Family Apple 5 GPU Family Apple 4 GPU Family Apple 3 GPU Family Apple 2 GPU Family Apple 1 GPU Family Mac 2 GPU Family Mac 1 GPU Family Common 3 GPU Family Common 2 GPU Family Common 1 macOS GPU Family 2 v1 macOS GPU Family 1 v4 macOS GPU Family 1 v3 macOS GPU Family 1 v2 macOS GPU Family 1 v1 [mvk-info] Created VkInstance for Vulkan version 1.1.224, as requested by app, with the following 3 Vulkan extensions enabled: VK_KHR_surface v25 VK_EXT_debug_utils v2 VK_EXT_metal_surface v1 [mvk-info] Using emulation for Vulkan semaphores. [mvk-info] Created VkDevice to run on GPU Apple M1 with the following 3 Vulkan extensions enabled: VK_KHR_driver_properties v1 VK_KHR_sampler_mirror_clamp_to_edge v3 VK_KHR_swapchain v70 [mvk-info] Created 3 swapchain images with initial size (1280, 720) and contents scale 1.0 for screen Built-in Retina Display. 2022-10-01 10:30:13.816 Cemu[2780:82761] IMKClient Stall detected, please Report your user scenario attaching a spindump (or sysdiagnose) that captures the problem - (imkxpc_bundleIdentifierWithReply:) block performed very slowly (1.16 secs). [mvk-error] VK_ERROR_INITIALIZATION_FAILED: Render pipeline compile failed (Error code 3): output of type float4 is not compatible with a MTLPixelFormatRGBA32Uint color attachement.. [mvk-error] VK_ERROR_INITIALIZATION_FAILED: Render pipeline compile failed (Error code 3): output of type float4 is not compatible with a MTLPixelFormatRGBA16Uint color attachement.. [mvk-info] Created 3 swapchain images with initial size (1280, 720) and contents scale 1.0 for screen Built-in Retina Display. [mvk-error] VK_ERROR_INITIALIZATION_FAILED: Render pipeline compile failed (Error code 3): output of type float4 is not compatible with a MTLPixelFormatRGBA32Uint color attachement.. [mvk-error] VK_ERROR_INITIALIZATION_FAILED: Render pipeline compile failed (Error code 3): output of type float4 is not compatible with a MTLPixelFormatRGBA16Uint color attachement.. Trace/BPT trap: 5! Error: signal 5:

Saving session... ...copying shared history... ...saving history...truncating history files... ...completed.

[Process completed]