search

COVESA / dlt-daemon

Diagnostic Log and Trace.
https://covesa.github.io/dlt-daemon/
Mozilla Public License 2.0
381 stars 294 forks source link

Save file may cause mem leak #655

Open hush-soul opened 4 months ago

hush-soul commented 4 months ago

I tested it on both qnx and linux use dlt-daemon -d -c /etc/dlt.conf dlt-logstorage-ctrl -c 1 -C /etc/dlt.conf -p /etc/dltlog dlt-example-user -d 5 -n 10000 Hello123

logstorage

hush-soul commented 4 months ago

Every time it is stored for a period of time, the memory usage will increase. Even if the storage quantity has reached the upper limit, old files will be deleted and new files will be stored.

qnx dlt_leak_1 dlt_leak_2 dlt_leak_3

hush-soul commented 4 months ago

linux leak_linux

hush-soul commented 4 months ago

Initially, I used the log interface of ap autorsa for testing. I created 15 processes and sent the log every 20ms. The memory usage increased wirelessly and very quickly.

In order to rule out AP problems, I used dlt-example-user to test, and the same problem also exists. It is currently under investigation. I would like to ask if there have been similar problems before.

hush-soul commented 4 months ago

I observed a phenomenon, dlt-daemon stack = 32kb in qnx, so the memory increases by 32kb each time

hush-soul commented 4 months ago

I found that this problem occurs when only performing the first and third steps 1.dlt-daemon -d -c /etc/dlt.conf 2.//dlt-logstorage-ctrl -c 1 -C /etc/dlt.conf -p /etc/dltlog 3.dlt-example-user -d 5 -n 10000 Hello123

hush-soul commented 4 months ago

I found that this problem occurs when only performing the first and third steps 1.dlt-daemon -d -c /etc/dlt.conf 2.//dlt-logstorage-ctrl -c 1 -C /etc/dlt.conf -p /etc/dltlog 3.dlt-example-user -d 5 -n 10000 Hello123

This test is wrong. When the storage is not set, it will stop when it reaches the RingbufferMaxsize. It does not stop when the storage is set.

hush-soul commented 4 months ago

I did some testing on Linux using valgrind Using 15 processes, each process sends a log every 20ms. The results are shown in Figure 4.

time:16h

4

hush-soul commented 4 months ago

So I looked at dlt_offline_logstorage_behavior.c:494 I found that the function dlt_logstorage_close_file when using DLT_LOGSTORAGE_USE_GZIP does not call fclose, only gzclose

hush-soul commented 4 months ago

I don't know if there are other considerations. So I just modified it myself and tested it. @minminlittleshrimp Modification 1: 1 Modification 2: 2

hush-soul commented 4 months ago

The results are as follows: On the left, 3

there are other places I may continue to test

hush-soul commented 4 months ago

dlt_logstorage_storage_dir_info Move the allocation outside the loop Seems to have solved the problem image

hush-soul commented 4 months ago

image

hush-soul commented 4 months ago

solvedsolved

minminlittleshrimp commented 3 months ago

Hello @hush-soul Thanks for raising your concern. May I ask for the fix, is it delivered in any PR yet?

hush-soul commented 3 months ago

Hello @hush-soul Thanks for raising your concern. May I ask for the fix, is it delivered in any PR yet?

Of course, you can check PR #666

minminlittleshrimp commented 3 months ago

Thank you for providing the fix, we will review. For now I would like to reopen this issue and will close together when the fix merged.