search

fhunleth / elixir_ale

Interact with hardware in Elixir - GPIOs, I2C and SPI
Apache License 2.0
344 stars 44 forks source link

Memory Leak #76

Closed alanj853 closed 2 years ago

alanj853 commented 4 years ago

Hi,

I am just letting you know that I have found a memory leak when using this application on a raspberry PI 3.

Background

My company has a product with architecture very similar to that of a raspberry PI and we are required to read GPIOs at regular intervals. We we reading every 10 seconds, and found that the memory was leaking, albeit very slowly. Now, we fixed our issue by actually just reading once and then caching the read value (we actually just needed one reading).

The Current Issue

As I mentioned, our device is very similar to that of a PI, and so I got a PI and tried to reproduce it there as I thought it would be easier for sharing my findings and for others to reproduce if they wanted. I also increased the rate at which I read the GPIOs to speed up the test. It is the "system" memory in the :erlang.memory() that increases, and never seems to be released (albeit in this test, the "processes" increases as well). I created a test project for anyone who wants to observe this: https://github.com/alanj853/gpio_bug Simply run: mix deps.get && iex -S mix iex> Reader.start_link()

This will start of reading the GPIOs every second, and writes the memory to a file (every 3 mins) for later analysis.

After leaving for about 30 mins, you can run the following to stop the reading of the GPIOs, and observe how the memory is never released: iex> Reader.stop_read()

After leaving for 30 mins of so, you can see the memory is never released.

Here is an output I collected earlier on:

+----------------------+-----------+---------------+----------+--------+----------+--------+---------+--------+ | Total | Processes | ProcessesUsed | System | Atom | AtomUsed | Binary | Code | ETS | +----------------------+-----------+---------------+----------+--------+----------+--------+---------+--------+ | 14692296 | 3413200 | 3411652 | 11279096 | 387413 | 378244 | 47760 | 6191273 | 346656 | | 14713104 | 3412736 | 3411092 | 11300368 | 387413 | 378244 | 47712 | 6191273 | 346520 | | 14793048 | 3446576 | 3444932 | 11346472 | 387413 | 378244 | 47664 | 6191273 | 346520 | | 14851064 | 3458432 | 3456788 | 11392632 | 387413 | 378244 | 47664 | 6191273 | 346552 | | 14873584 | 3434872 | 3433324 | 11438712 | 387413 | 378244 | 47664 | 6191273 | 346552 | | 14937416 | 3452616 | 3450972 | 11484800 | 387413 | 378244 | 47664 | 6191273 | 346552 | | 14997800 | 3466920 | 3465276 | 11530880 | 387413 | 378244 | 47664 | 6191273 | 346552 | | 15034800 | 3457832 | 3456116 | 11576968 | 387413 | 378244 | 47664 | 6191273 | 346552 | | 15087088 | 3464056 | 3462508 | 11623032 | 387413 | 378244 | 47664 | 6191273 | 346552 | | GPIO Reading stopped | | | | | | | | | | 15271072 | 3614692 | 3614692 | 11656380 | 387413 | 378244 | 47688 | 6191273 | 346504 | | 15293016 | 3636236 | 3636236 | 11656780 | 387413 | 378244 | 48144 | 6191273 | 346504 | | 15304472 | 3647580 | 3647580 | 11656892 | 387413 | 378244 | 48144 | 6191273 | 346504 | | 15269912 | 3613236 | 3613236 | 11656676 | 387413 | 378244 | 47688 | 6191273 | 346688 | | 15295432 | 3638436 | 3638436 | 11656996 | 387413 | 378244 | 48144 | 6191273 | 346552 | | 15279800 | 3623220 | 3623220 | 11656580 | 387413 | 378244 | 47784 | 6191273 | 346552 | | 15295376 | 3638436 | 3638436 | 11656940 | 387413 | 378244 | 48144 | 6191273 | 346552 | | 15278368 | 3621372 | 3621372 | 11656996 | 387413 | 378244 | 48144 | 6191273 | 346552 | | 15286472 | 3629932 | 3629932 | 11656540 | 387413 | 378244 | 47688 | 6191273 | 346552 | | 15273304 | 3616820 | 3616820 | 11656484 | 387413 | 378244 | 47688 | 6191273 | 346552 | | 15297552 | 3640636 | 3640540 | 11656916 | 387413 | 378244 | 48048 | 6191273 | 346552 | | 15317536 | 3660628 | 3660628 | 11656908 | 387413 | 378244 | 48048 | 6191273 | 346552 | | 15283488 | 3627148 | 3627148 | 11656340 | 387413 | 378244 | 47592 | 6191273 | 346552 | +----------------------+-----------+---------------+----------+--------+----------+--------+---------+--------+

Perhaps this isn't an issue, and maybe the way I am reading and shutting down is not the best approach, but I thought I'd share it anyways just in case.

More Info

Elixir and OTP Version: IEx 1.10.3 (compiled with Erlang/OTP 21) elixir_ale version: 1.2.1 raspberry PI version: 3 Kernel: Linux raspberrypi 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l GNU/Linux

fhunleth commented 4 years ago

Thanks for investigating this and for sending the report with an example. I can't tell you how much I appreciate this.

I have not seen this memory leak, but it's really curious since there's not much going on. In my usage, I have long lived GenServers that hold on to GPIOs pretty much forever, so I'm sure that I'm not exercising this case.

I've pretty much stopped maintaining ElixirALE and focused my efforts on Elixir Circuits. Would you be ok in switching to circuits_gpio and rerunning the experiment? If you're only reading the GPIO infrequently, I don't think the switch is too hard. Here's the guide: https://github.com/elixir-circuits/circuits_gpio/blob/main/PORTING.md#code-modifications.

Assuming that the memory leak is in Elixir Circuits as well, I'd much prefer to debug it there. Elixir Circuits is also lighter weight, so I'd expect a memory leak, if it exists, to be smaller.