search

KhronosGroup / SPIRV-Cross

SPIRV-Cross is a practical tool and library for performing reflection on SPIR-V and disassembling SPIR-V back to high level languages.
Apache License 2.0
2.07k stars 565 forks source link

Memory Usage Jumps by 200MB upon first invocation of the compiler (Debug build) #1234

Open devshgraphicsprogramming opened 4 years ago

devshgraphicsprogramming commented 4 years ago

This is how we use the SPIR-V API https://github.com/buildaworldnet/IrrlichtBAW/blob/974d0ff631f65e92d662aca5582e33d6ac4e5c49/src/irr/asset/IGLSLCompiler.cpp#L10

Looking at the Visual Studio memory analyzer, my memory usage jumps by about 200 MB as soon as the SPIR-V compliation function is invoked.

I also used heap snapshots, on breakpoints before and after and the compile, fist time round I can see some allocations from static variables (such as string maps for different stuff like variables, etc.) but they don't match 200MB increase exactly.

Second time I come to compile, I don't get the 200MB jump.

I'm fine with a 200MB memory usage while compiling, but there should be a way to "fully deinitialize" SPIR-V cross to reclaim the 200MB after I'm done compiling all my shaders.

HansKristian-Work commented 4 years ago

Are you sure this is the correct repository? That snippet is invoking shaderc.

devshgraphicsprogramming commented 4 years ago

We use SPIR-V cross as a whole, with all its dependencies (incl shaderc) to create SPIR-V from Vulkan-dialect GLSL, remap descriptor sets into OpenGL objects bindings and push constants to a unbacked plain-uniform struct, then downcompile the SPIR-V to OpenGL dialect GLSL.

So you could be right by stating that its shaderc that actually leaks.

HansKristian-Work commented 4 years ago

I'd like to get a confirmation SPIRV-Cross in isolation is the culprit before digging into this, with at least a SPIR-V I can test with.

Another thing to consider is that the memory reported may simply be virtually allocated when it backs the process with 200 MB, but when app frees, the malloc implementation just keeps the memory around for other allocations. Not exactly sure how the VS memory analyzer deals with that though, but something to consider.

devshgraphicsprogramming commented 4 years ago

Not exactly sure how the VS memory analyzer deals with that though, but something to consider.

I believe it shows physical not virtual memory.

Heap increases by 10 MB (that was from just one static string map), but overall 200MB, it might be related to enormous amounts of static variables in shaderc.