Render large scene crashed with Vulkan

[X-skyer]

Describe the bug Hi, I have use bgfx to render lots of scenes and all of them are runing perfect. Today I got nvidia's bistro scene, which contains hundreds of textures and thousands of mesh, in opengl and directx, the render is OK, but in vulkan, the renderer crashed after loading all the textures.

The error code is maxMemoryAllocationCount when loading, and after loading got this error and crashed: C:\Test\3rdparty\bgfx\src\renderer_vk.cpp (3732): BGFX ASSERT vkAllocateDescriptorSets(m_device, &dsai, &descriptorSet); VK error 0xc4642878: VK_ERROR_OUT_OF_POOL_MEMORY C:\Test\3rdparty\bgfx\src\renderer_vk.cpp (3732): BGFX FATAL 0x00000000: vkAllocateDescriptorSets(m_device, &dsai, &descriptorSet); VK error 0xc4642878: VK_ERROR_OUT_OF_POOL_MEMORY

I've tested if I skip the first 40 textures when loading the scene(Total number of texture is 271), then the rendering is OK, but adding one more texture, will crash.

I also tested if I replace all the texture with a 16X16 texture, and the renderer also crashes. The vram usage is 2.1GB/10GB, which is enough.

[MarioSieg]

I also encountered this error.

[MarioSieg]

Any updates on this?

[pezcode]

Any updates on this?

No updates on fixing it, but I can give you a bit of background 😃 Maybe someone else is bored and feels like tackling this. This is a combination of two limitations:

• Vulkan by default only guarantees 4096 memory allocations and you're supposed to write your own sub-allocator (or use a 3rd party library), but the bgfx backend currently doesn't do that. Meaning, you can only create a total of 4k textures, buffers, etc. combined before the driver might error out. It would also greatly improve performance for buffer and texture updates. Just slapping something like VMA into bgfx would be counter-productive I believe - half the features aren't needed, and the suballocator should be re-usable for D3D12.
• Draw calls in Vulkan use "descriptor sets" to bind resources, and those also need to be allocated from a pool. Once that pool is empty, bgfx currently doesn't move to a new pool but simply asserts. This one is a bit simpler - sets should be cached to reduce allocations, and once a pool is full, a new larger one should be created.

You can try to work around the allocator issue by merging resources where possible, e.g. put all your meshes into one big vertex buffer and submit draws with a non-0 start vertex/index.

[bkaradzic]

It would also greatly improve performance for buffer and texture updates. Just slapping something like VMA into bgfx would be counter-productive I believe - half the features aren't needed, and the suballocator should be re-usable for D3D12.

Yes, we don't want VMA, I'm rewriting NonLocalAllocator that's currently used for dynamic resources, and new one will be more generalized and able to handle other resources too.

[MarioSieg]

What's the state of this issue? I'm currently reworking our engine renderer and I'm thinking to move to The Forge because we have huge cities with lots of textures :/

[bkaradzic]

I'm currently reworking our engine renderer and I'm thinking to move to The Forge

No ETA, you should switch to The Forge.

[julianxhokaxhiu]

Is there any update on this front? I'm also facing a similar issue on rendering a large world scene. Thanks

[kalmard0]

I ran into the same problem and found that increasing the value of MAX_DESCRIPTOR_SETS seems to alleviate the issue. I have no idea what additional side-effects that might have and I haven't tested it too much.