bkaradzic / bgfx

Cross-platform, graphics API agnostic, "Bring Your Own Engine/Framework" style rendering library.
https://bkaradzic.github.io/bgfx/overview.html
BSD 2-Clause "Simplified" License
15.11k stars 1.95k forks source link

Crash using Metal renderer when using RGB5A1 textures #3344

Open zacbrannelly opened 2 months ago

zacbrannelly commented 2 months ago

Describe the bug

In the Metal renderer, a simple call to createTexture2D to create a RGB5A1 texture has side effects that breaks reset, specifically when the depth buffer is resized.

After calling createTexture2D with a pixel format that uses texture swizzle which applies swizzle that isn't just R -> R, G -> G, B -> B, A -> A (for RGB5A1 it is R -> B, G -> G, B -> R, A -> A), then I get the following error when calling reset, which I call after window resize:

-[MTLTextureDescriptorInternal validateWithDevice:]:1357: failed assertion `Texture Descriptor Validation
Texture swizzling is not compatable with MTLTextureUsageRenderTarget

To Reproduce

bgfx::createTexture2D(100, 100, false, 1, RGB5A1, BGFX_TEXTURE_NONE, NULL);
bgfx::reset(1920, 1080, 0);

Expected behavior

reset to complete without hitting an assertion.

Additional context

I can fix the problem by editing the SwapChainMtl::resize function in renderer_mtl.mm, where I change this:

if (s_renderMtl->m_hasPixelFormatDepth32Float_Stencil8)
{
    desc.pixelFormat = MTLPixelFormatDepth32Float_Stencil8;
}

To this:

if (s_renderMtl->m_hasPixelFormatDepth32Float_Stencil8)
{
    desc.pixelFormat = MTLPixelFormatDepth32Float_Stencil8;
    desc.swizzle = { MTLTextureSwizzleRed, MTLTextureSwizzleGreen, MTLTextureSwizzleBlue, MTLTextureSwizzleAlpha };
}

Happy to make a PR but cognisant of the fact I'm not entirely sure what/if there are any other issues this would cause.