Open PaulMartinsen opened 7 months ago
Hi @PaulMartinsen, thanks for reaching out to us concerning this problem. Let me first address your questions.
heap_caps_add_region_with_caps()
in esp_heap_caps_init.h
. That can be a workaround that gives you the choice of where to allocate.I am currently working on an alternative approach to the startup stack problem that will result in creating a single heap per memory region. That will solve the issue you are facing at the moment.
Thanks @SoucheSouche for your quick and helpful comments. Particularly for clarifying where heap 3 comes from. Your new approach to join heaps 2 & 3 sounds great and a lot better than allocating into heap 3 explicitly (which would be gone with the new approach). I think this will be a big step forward for us.
When I enabled support for malloc
to use PSRAM (possibly a temporary work-around), I noticed there's an option to reserve a section of internal ram that can be used by the capability allocator and is not used by malloc
. This does go into heap 2/3. I don't think we will need this, but I did wonder if having an option to place this in "heap 4" would be straight forward and worthwhile.
Thanks again for your comments and work on the memory management.
@PaulMartinsen, no problem. we can leave this ticket open and I will link it to the merge request for this heap initialization update so it gets closed when the new implementation gets merged in master.
The modifications are quite big so it might take a little while before I can merge.
Also seeing similar issues - unable to start camera because it wants to make a ~30kb dma-capable allocation, but largest single block of dram is only ~28kb, even though >80kb free dram.
esp32 classic does not have this problem afaics because the smaller heaps are first, so get used first, leaving most of the final (largest) heap free.
esp32s3 has the largest heap first, so that gets (partially) used, leaving part of it, and the smaller heaps, free - but no single large block available.
Reordering the [equivalent DRAM] heaps such that the smaller ones are searched/used before the larger would be a huge win, I'm sure.
@SoucheSouche How is the status of this fix? https://github.com/espressif/tlsf/pull/2
@SoucheSouche How is the status of this fix? espressif/tlsf#2
@SoucheSouche When will esp-idf update tlsf submodule to get above fix? Will you update tlsf for release branches as well?
@AxelLin, the MR to update the tlsf submodule is created and under review now. No backport can be done for https://github.com/espressif/tlsf/pull/2 because backporting implies patching the ROM version of the TLSF and since this PR updates static inline functions, it simply can't be done.
@PaulMartinsen the new implementation of the heap init process is stalled because of ROM TLSF backport issue as well. In the mean time, I will update the heap init to sort the heaps by increasing size to fix your issue and backport this fix.
@PaulMartinsen, I created a fix that sorts the heap by increasing size. It is currently under review.
@AxelLin, the MR to update the tlsf submodule is created and under review now. No backport can be done for espressif/tlsf#2 because backporting implies patching the ROM version of the TLSF and since this PR updates static inline functions, it simply can't be done.
@SoucheSouche I don't quite understand this.
I notice the v5.4 branch includes the fix espressif/tlsf#2 . Now I'm using v5.2 branch. If rebuild my application using v5.4 branch and OTA firmware on the device, will it work?
@PaulMartinsen, I created a fix that sorts the heap by increasing size. It is currently under review.
@SoucheSouche How is the status now? Could you share the commit for this fix?
@AxelLin, the fix has been reviewed and will be merged soon. The commit will be made available when the work is merged and synced with GH.
Answers checklist.
General issue report
Please rethink the capability heap's allocation strategy.
Currently
malloc
appears to allocate from the largest heap first. This led to a fragmentation related allocation fault. It would seem better to place smaller objects in smaller heaps first by default. I think the largest heap is just the piece of contiguous memory left after statically scoped variables are placed by the complier.Details
Yesterday I ran out of heap memory. More precisely, I had about 100k free of
MALLOC_CAP_8BIT
capable memory, needed 36k but the largest free block was only 34k. I was sad.It appears the ESP32s3 (and probably other devices):
malloc
fills up the 200k ram first. This is why I was unable to obtain a 36k block of RAM.For reference, here's a summary of my memory disposition:
Questions
heap_caps_malloc
to place an object in heap 3 or 4 in the diagram above? I can't see any combination ofMALLOC_CAP_*
flags that would let me do this before heap 2 was too small for the allocation.