Fix Metal backend on OSX

[hugoam]

Metal backend has been found to fail with the following stacktrace:

-[MTLVertexDescriptorInternal newSerializedDescriptor]:832: failed assertion `None of the attributes set bufferIndex to 2, but MTLVertexDescriptor set buffer layout[2].stride(4).'
Process 87068 stopped

    thread #8, stop reason = signal SIGABRT
    frame #0: 0x00007fff5cf7be3e libsystem_kernel.dylib__pthread_kill + 10 libsystem_kernel.dylib__pthread_kill:
    -> 0x7fff5cf7be3e <+10>: jae 0x7fff5cf7be48 ; <+20>
    0x7fff5cf7be40 <+12>: movq %rax, %rdi
    0x7fff5cf7be43 <+15>: jmp 0x7fff5cf730b8 ; cerror_nocancel
    0x7fff5cf7be48 <+20>: retq
    Target 0: (00_cube_d) stopped.
    (lldb) bt
    thread #8, stop reason = signal SIGABRT
        frame #0: 0x00007fff5cf7be3e libsystem_kernel.dylib__pthread_kill + 10 frame #1: 0x00007fff5d0ba150 libsystem_pthread.dylibpthread_kill + 333
        frame #2: 0x00007fff5ced8312 libsystem_c.dylibabort + 127 frame #3: 0x00007fff5cea0368 libsystem_c.dylib__assert_rtn + 320
        frame #4: 0x00007fff39ad3f8d MetalMTLReportFailure + 778 frame #5: 0x00007fff39adc805 Metal-[MTLVertexDescriptorInternal newSerializedDescriptor] + 454
        frame #6: 0x00007fff39ad362f Metal-[MTLRenderPipelineDescriptorInternal newSerializedVertexDataWithFlags:error:] + 490 frame #7: 0x00007fff39ac1382 Metal-[MTLCompiler newRenderPipelineStateWithDescriptor:options:reflection:error:completionHandler:] + 153
        frame #8: 0x00007fff39aa43a4 Metal-[_MTLDevice newRenderPipelineStateWithDescriptor:options:reflection:error:] + 76 frame #9: 0x00000001018c619c 00_cube_dbgfx::mtl::Device::newRenderPipelineStateWithDescriptor(this=0x0000000106f16008, _descriptor=0x00000001034759e0, _options=2, _reflection=0x000070000b8aa0c8) at renderer_mtl.h:315
        frame #10: 0x00000001018be593 00_cube_dbgfx::mtl::RendererContextMtl::getPipelineState(this=0x0000000106f16000, _state=106319887, _rgba=0, _fbh=(idx = 65535), _numStreams='\x03', _vertexDecls=0x000070000b8aac50, _programIdx=11, _numInstanceData='\0') at renderer_mtl.mm:2010 frame #11: 0x00000001018b90cc 00_cube_dbgfx::mtl::RendererContextMtl::submit(this=0x0000000106f16000, _render=0x000000010c800440, _clearQuad=0x000000010f343e90, _textVideoMemBlitter=0x000000010f343e20) at renderer_mtl.mm:3849
        frame #12: 0x0000000101858d80 00_cube_dbgfx::Context::renderFrame(this=0x000000010c800040, _msecs=5000) at bgfx.cpp:1911 frame #13: 0x0000000101858c46 00_cube_dbgfx::renderFrame(_msecs=-1) at bgfx.cpp:1096
        frame #14: 0x0000000101859ee3 00_cube_dbgfx::Context::renderThread((null)=0x000000010c800240, (null)=0x000000010c800040) at bgfx_p.h:2714 frame #15: 0x00000001018fd783 00_cube_dbx::Thread::entry(this=0x000000010c800240) at thread.cpp:294
        frame #16: 0x00000001018fd72d 00_cube_dbx::ThreadInternal::threadFunc(_arg=0x000000010c800240) at thread.cpp:89 frame #17: 0x00007fff5d0b76c1 libsystem_pthread.dylib_pthread_body + 340
        frame #18: 0x00007fff5d0b756d libsystem_pthread.dylib_pthread_start + 377 frame #19: 0x00007fff5d0b6c5d libsystem_pthread.dylibthread_start + 13

For now this can be worked around by forcing the renderer to OpenGL, but in the future we want to support the Metal backend.

[jdryg]

I accidentally ended up here after github suggested to take a look at mud :)

I solved a similar issue with Metal a couple of weeks ago. In my case the problem seemed to be that I was setting a vertex stream which wasn't used by the shader. More specifically, I was using 3 vertex streams (pos, color and uv) and all of them were used by the most common shader. But once I used a shader which didn't use the uv stream, I got that error from Metal. Solved by moving the uv stream binding inside the specific case.

See: https://github.com/jdryg/vg-renderer/commit/23e2221c7e72058547557f17ed138beb4423c5ed

Hope it helps :)

[hugoam]

Well sounds like it might actually come from vg-renderer then :) Sadly I don't have an OSX device myself to check. Thanks for the heads up!