Hitting non-empty VmaDeviceMemoryBlock assert

[rumblehhh]

We're hitting the metadata->IsEmpty() assert on shutdown inside VmaDeviceMemoryBlock::Destroy however VmaBlockMetadata_TLSF::DebugLogAllAllocations() isn't printing a list of unfreed allocations - we've defined VMA_DEBUG_LOG. I see that VmaBlockMetadata_TLSF::DebugLogAllAllocation is getting skipped due to the !block->IsFree() check.

I've copied the VMA specific part of our logs below. The number of allocations seems to line up perfectly with the number of allocations freed. Are we missing anything obvious?

Note that I modified the allocation/freed VMA_DEBUG_LOGs to include the allocation handle to help debug this but all seems fine.

We've (PVRCarbon: https://developer.imaginationtech.com/pvrcarbon/) been using v3.0.0 since March (Vma in general a lot longer) and we've not hit this issue before. I also tried v3.0.1 this morning and see the same behaviour.

It looks like its a particular path is being taken to cause this. Have you got any suggestions?

vmaCreateAllocator vmaCreateAllocator vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=7680 Created new block #0 Size=33554432, allocation=140737127085760 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=768 Returned from existing block #0, allocation: 140737127085840 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=398272 Returned from existing block #0, allocation: 140737127085920 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=120348 Returned from existing block #0, allocation: 140737127086000 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=120348 Returned from existing block #0, allocation: 140737127086080 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=2368 Returned from existing block #0, allocation: 140737127086160 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=432 Returned from existing block #0, allocation: 140737127086240 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=432 Returned from existing block #0, allocation: 140737127086320 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=35552 Returned from existing block #0, allocation: 140737127086400 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4932 Returned from existing block #0, allocation: 140737127086480 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4932 Returned from existing block #0, allocation: 140737127086560 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=35616 Returned from existing block #0, allocation: 140737127086640 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4932 Returned from existing block #0, allocation: 140737127086720 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4932 Returned from existing block #0, allocation: 140737127086800 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=56800 Returned from existing block #0, allocation: 140737127086880 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16752 Returned from existing block #0, allocation: 140737127086960 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16752 Returned from existing block #0, allocation: 140737127087040 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=768 Returned from existing block #0, allocation: 140737127087120 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=72 Returned from existing block #0, allocation: 140737127087200 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=72 Returned from existing block #0, allocation: 140737127087280 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=5120 Returned from existing block #0, allocation: 140737127087360 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1368 Returned from existing block #0, allocation: 140737127087440 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1368 Returned from existing block #0, allocation: 140737127087520 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=5120 Returned from existing block #0, allocation: 140737127087600 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1368 Returned from existing block #0, allocation: 140737127087680 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1368 Returned from existing block #0, allocation: 140737127087760 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=40320 Returned from existing block #0, allocation: 140737127087840 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4188 Returned from existing block #0, allocation: 140737127087920 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4188 Returned from existing block #0, allocation: 140737127088000 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4194304 Returned from existing block #0, allocation: 140737127088080 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1048576 Returned from existing block #0, allocation: 140737127088160 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=262144 Returned from existing block #0, allocation: 140737127088240 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=65536 Returned from existing block #0, allocation: 140737127088320 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16384 Returned from existing block #0, allocation: 140737127088400 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4096 Returned from existing block #0, allocation: 140737127088480 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1024 Returned from existing block #0, allocation: 140737127088560 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=256 Returned from existing block #0, allocation: 140737127088640 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=64 Returned from existing block #0, allocation: 140737127088720 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16 Returned from existing block #0, allocation: 140737127088800 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4 Returned from existing block #0, allocation: 140737127088880 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1048576 Returned from existing block #0, allocation: 140737127088960 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=262144 Returned from existing block #0, allocation: 140737127089040 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=65536 Returned from existing block #0, allocation: 140737127089120 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16384 Returned from existing block #0, allocation: 140737127089200 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4096 Returned from existing block #0, allocation: 140737127089280 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1024 Returned from existing block #0, allocation: 140737127089360 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=256 Returned from existing block #0, allocation: 140737127089440 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=64 Returned from existing block #0, allocation: 140737127089520 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16 Returned from existing block #0, allocation: 140737127089600 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4 Returned from existing block #0, allocation: 140737127089680 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=2097152 Returned from existing block #0, allocation: 140737127089760 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=524288 Returned from existing block #0, allocation: 140737127089840 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=131072 Returned from existing block #0, allocation: 140737127089920 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=32768 Returned from existing block #0, allocation: 140737127090000 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=8192 Returned from existing block #0, allocation: 140737127090080 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=2048 Returned from existing block #0, allocation: 140737127090160 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=512 Returned from existing block #0, allocation: 140737127090240 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=128 Returned from existing block #0, allocation: 140737127090320 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=32 Returned from existing block #0, allocation: 140737127090400 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=8 Returned from existing block #0, allocation: 140737127090480 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4 Returned from existing block #0, allocation: 140737127090560 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16384 Returned from existing block #0, allocation: 140737127090640 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=16384 Returned from existing block #0, allocation: 140737127090720 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=356088 Returned from existing block #0, allocation: 140737127090800 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=88800 Returned from existing block #0, allocation: 140737127090880 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=22000 Returned from existing block #0, allocation: 140737127090960 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=5400 Returned from existing block #0, allocation: 140737127091040 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=1300 Returned from existing block #0, allocation: 140737127091120 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=288 Returned from existing block #0, allocation: 140737127091200 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=72 Returned from existing block #0, allocation: 140737127091280 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=12 Returned from existing block #0, allocation: 140737127091360 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4 Returned from existing block #0, allocation: 140737127091440 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=131072 Returned from existing block #0, allocation: 140737127091520 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=20 Returned from existing block #0, allocation: 140737127091600 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=20 Returned from existing block #0, allocation: 140737127091680 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=20 Returned from existing block #0, allocation: 140737127091760 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090800 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090880 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090960 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091040 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091120 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091200 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091280 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091360 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091440 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091520 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=196596 Returned from existing block #0, allocation: 140737127091520 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127085920 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086080 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086160 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086320 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086400 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086560 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086640 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086800 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086880 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087040 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087120 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087280 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087360 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087520 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087600 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087760 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087840 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088000 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088080 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088160 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088240 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088320 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088400 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088480 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088560 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088640 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088720 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088800 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088880 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127088960 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089040 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089120 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089200 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089280 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089360 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089440 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089520 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089600 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089680 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089760 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089840 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127089920 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090000 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090080 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090160 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090240 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090320 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090400 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090480 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090560 vmaAllocateMemoryForBuffer AllocateMemory: MemoryTypeIndex=0, AllocationCount=1, Size=4587520 Returned from existing block #0, allocation: 140737127090560 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090560 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091680 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091760 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127085760 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127085840 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091520 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127091600 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090640 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127090720 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086000 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086240 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086480 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086720 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127086960 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087200 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087440 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087680 vmaFreeMemory Freed from MemoryTypeIndex=0, allocation: 140737127087920 vmaDestroyAllocator

[medranSolus]

Hello and thanks for reporting that! Could you provide us with JSON containing allocator statistics about current memory usage during that situation? You can obtain it via function vmaBuildStatsString() with detailedMap parameter set to true (just call it before destruction of allocator). Also if you would specify which device block is causing said problem that would help us greatly in finding the reason behind all of this (value of m_Id of VmaDeviceMemoryBlock). Marek Machliński, Developer Technology Engineer Intern, AMD

[rumblehhh]

Hi Marek, Please see the following json stats. m_Id is 0 as it seems we only use a single block.

{ "General": { "API": "Vulkan", "apiVersion": "1.3.0", "GPU": "SwiftShader Device (LLVM 10.0.0)", "deviceType": 4, "maxMemoryAllocationCount": 4096, "bufferImageGranularity": 4096, "nonCoherentAtomSize": 256, "memoryHeapCount": 1, "memoryTypeCount": 1 }, "Total": { "BlockCount": 1, "BlockBytes": 33554432, "AllocationCount": 0, "AllocationBytes": 0, "UnusedRangeCount": 4, "UnusedRangeSizeMin": 7776, "UnusedRangeSizeMax": 32706928 }, "MemoryInfo": { "Heap 0": { "Flags": ["DEVICE_LOCAL"], "Size": 2147483648, "Budget": { "BudgetBytes": 1717986918, "UsageBytes": 33554432 }, "Stats": { "BlockCount": 1, "BlockBytes": 33554432, "AllocationCount": 0, "AllocationBytes": 0, "UnusedRangeCount": 4, "UnusedRangeSizeMin": 7776, "UnusedRangeSizeMax": 32706928 }, "MemoryPools": { "Type 0": { "Flags": ["DEVICE_LOCAL", "HOST_VISIBLE", "HOST_COHERENT", "HOST_CACHED"], "Stats": { "BlockCount": 1, "BlockBytes": 33554432, "AllocationCount": 0, "AllocationBytes": 0, "UnusedRangeCount": 4, "UnusedRangeSizeMin": 7776, "UnusedRangeSizeMax": 32706928 } } } } }, "DefaultPools": { "Type 0": { "PreferredBlockSize": 268435456, "Blocks": { "0": { "MapRefCount": 0, "TotalBytes": 33554432, "UnusedBytes": 33554432, "Allocations": 0, "UnusedRanges": 4, "Suballocations": [ {"Offset": 0, "Type": "FREE", "Size": 7776}, {"Offset": 7776, "Type": "FREE", "Size": 831840}, {"Offset": 839616, "Type": "FREE", "Size": 7888}, {"Offset": 847504, "Type": "FREE", "Size": 32706928} ] } }, "DedicatedAllocations": [ ] } }, "CustomPools": { } }

[medranSolus]

Hello and sorry for long time in responses. I've reproduced your application allocations and de-allocations in same order but seems I've did not hit any library assert, nor achieved same situation you posted in JSON dump (thanks for that, it helped us narrow what could be the problem here). My suggestion here would be to enable VMA_HEAVY_ASSERT() (by just simply defining it to VMA_ASSERT() for ex.) to perform additional checks inside library that can catch more complex scenarios. Also are you using VMA in multithreaded application? If so then what method of synchronization are you using, your own or the one from library itself (see VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT)? Situation from JSON dump you posted seems like some race conditions occurred that trashed internal state of library, so if you could check that I'd be greatful. Marek Machliński, Developer Technology Engineer Intern, AMD

[rumblehhh]

No problem! VMA_HEAVY_ASSERT didn't trigger during the application run - I checked that VmaIsPow2 was called inside VmaAlignUp so VMA_HEAVY_ASSERT was definitely enabled. The application itself is multithreaded but all vulkan specific calls are made on a single thread so from VMAs perspective its single threaded. We don't use VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT i.e. we let VMA handle its own sync.

I had a play around with disabling some of the additional vma debugging features we use and the issue went away. In debug builds we currently enable VMA_DEBUG_INITIALIZE_ALLOCATIONS, VMA_DEBUG_DETECT_CORRUPTION and set VMA_DEBUG_MARGIN=4. If I disable all of these the problem goes away. If I enable only VMA_DEBUG_MARGIN=4 the problem comes back so it seems related to its usage. Could VMA be trashing its own state when using this flag?

[medranSolus]

Hi and thanks for checking that! I've tested said scenario with same settings but sadly could not reproduce the same bug you caught. Maybe you are using some other options that interfere with debug margin that we are not aware of? For now the best workaround I could suggest for you is to disable VMA_DEBUG_MARGIN till we could sort this problem out. Marek Machliński, Developer Technology Engineer Intern, AMD

[rumblehhh]

No problem.

Maybe you are using some other options that interfere with debug margin that we are not aware of

We're not explicitly enabling any other features but we will set VmaAllocatorCreateInfo::flags based on the implementations supported features.

If it helps I'm using Linux + clang-14. Are there any additional logs I can gather which might help understand when/why m_NullBlock->offset (this seems to be what determines whether the assert in question is thrown) is being modified incorrectly?

For now the best workaround I could suggest for you is to disable VMA_DEBUG_MARGIN till we could sort this problem out.

If we can figure out the root cause that'd be preferable as using VMA_DEBUG_MARGIN + VMA_DEBUG_INITIALIZE_ALLOCATIONS/VMA_DEBUG_DETECT_CORRUPTION have helped us debug other issues previously.

[medranSolus]

Hi, the problem that is appearing here is caused by improper merging of free blocks during their deletion in your case. When creating new allocation with debug margin there is block placed behind the actual allocation indicating requested margin.

This seems to be causing the whole merging problem. I could dig deeper to that if you could share with me 3 more logs (same as previous JSON dump):

• first one before you start freeing up memory,
• second one after first allocation is made after some deletion,
• third one when you made second allocation after some more deletions.

Maybe that would indicate what is happening next, as it would show me the layout of memory before corruption. Marek Machliński, Developer Technology Engineer Intern, AMD

[rumblehhh]

We have the following pattern:

1. Lots of allocations
2. Lots of frees
3. One allocation
4. Lots of frees

I'm attaching:

• a log with the allocations/frees
• a log with a json dump before freeing memory (at the start of 2)
• a log with a json dump after first allocation after deletions (just after 3)
• a log with a json dump after first deletion after allocation after deletions (just after the first deletion in 4)

after_first_allocation_after_deletion.txt after_first_deletion_after_allocation_after_deletions.txt before_freeing_memory.txt vma_log.txt

[medranSolus]

Hi and thanks for that logs, they really helped us! We've created branch fix-debug-margin with proposed fix for that issue, if you could check it out in your situation, we will merge it into main branch. Marek Machliński, Developer Technology Engineer Intern, AMD

[rumblehhh]

Thanks for the proposed fix! Unfortunately I still see the same assert :(.

If I additionally enable VMA_HEAVY_ASSERT by setting it to VMA_ASSERT I then hit the following assert virtual bool VmaBlockMetadata_TLSF::Validate() const: Assertion `0 && "Validation failed: " "calculatedFreeSize == GetSumFreeSize()"' failed.

[medranSolus]

Hi, sorry to hear that. About new validation assert, I fixed it now on the same branch. Could you send me JSON dumps from before starting frees (after point 1 from your previous post) and before deleting allocator (after point 4)? With previous fix I've deleted some corner case in allocator, sadly not related to your issue, but real one can become more apparent now. Thanks for your help so far! Marek Machliński, Developer Technology Engineer Intern, AMD

[rumblehhh]

Sorry for the delay - I've been OoO for the last week. No problem glad I can help. Please see attached the attached files as requested: before_freeing_memory_2.txt before_freeing_allocator.txt