numworks / epsilon

Modern graphing calculator operating system.
https://www.numworks.com/resources/engineering/software/
1.7k stars 461 forks source link

Add GC library to python #2202

Open yannis300307 opened 1 month ago

yannis300307 commented 1 month ago

Problem you'd like to fix

Sometimes, we need to allocate a lot of memory, but if the garbage collector doesn't delete old variables quickly enough, we can run into a memory allocation error.

Describe the solution you'd like

Add the gc lib to python to free the memory.

Describe alternatives you've considered

Make the del keyword forcing the garbage collection.

adri1numworks commented 4 weeks ago

Thanks for your suggestion. The gc lib is a bit too advanced for our students needs, we don't plan on adding it in future versions at the moment.

Make the del keyword forcing the garbage collection.

Do you have examples where del function is not working as you want it to ?

yannis300307 commented 4 weeks ago

I've made a project in which I have to store images as bytes. Sometimes, the memory is not cleared quickly enough between the images' loading despite the use of del. Moreover, the memory is not always cleared between programs executions. You can see this project here. It makes the emulator bugging, and it crashes on actual hardware (N0110) after 1 or 2 lignes rendered.

EmilieNumworks commented 4 weeks ago

MicroPython's garbage collector is triggered when the allocator runs out of memory. There shouldn't be such things as "it doesn't delete old variables quickly enough". The objects in the pool aren't deleted if some pointers still refer to them... What you're describing might be a bug but I cannot reproduce it (neither on the emulator nor the actual devices). Could you give the versions of your device or a video of the emulator bugging?

yannis300307 commented 4 weeks ago

I've a N0110 and the OS version is 22.2.0. I will send a video at the evening.

Volcanio18 commented 3 weeks ago

Hi, I've also encountered the same problem. You can easily reproduce this bug using this program :

a="" while True; ‎ ‎ ‎ ‎ a+="0" ‎ ‎ ‎ ‎ if len(a)%100==0: print(len(a))

You will see that every time you run this script it will run out of memory with "a" having reached a different length each time, because the ram was not cleared proprely.

yannis300307 commented 3 weeks ago

In this case, you never clear a during the execution, so you will always run into a MemoryAllocationError.

yannis300307 commented 3 weeks ago

I've reproduced the bug in this video: https://github.com/numworks/epsilon/assets/52321382/cb96bfb5-b80f-41d6-8ac4-cac5d3890c8b

As you can see, I clear a at the end of my for loop with del but the memory is not correctly cleared and it causes the error.

EmilieNumworks commented 3 weeks ago

Hi, I think @Volcanio18 is pointing at a heap leakage which I can indeed observe by enabling the MICROPY_MEM_STATS macro. @yannis300307 you implying that the del keyword doesn't work as intended. I'm opening issues internally for both topic and will keep you posted.