WildernessLabs / Meadow_Issues

Public repo for bugs and issues with Meadow
15 stars 0 forks source link

Memory leak/fragmentation on network #773

Open duduita opened 1 month ago

duduita commented 1 month ago

Describe the bug There's an increase over time of total memory used over WiFi, it looks like a small leak, something like ~20 bytes per minute when running Cultivar sending metrics every minute. There's also an increase in the number of total free blocks, which indicates a memory fragmentation over time much higher than over Ethernet.

To Reproduce Just run Cultivar app with health metrics enabled, or another network app that uses PostAsync() or similar API, and keep tracking the total_memory_free_blocks and total_free_memory in the Meadow.Cloud.

Expected behavior The total_free_memory just oscillates around a value, and does not keep decreasing over time until leaks on almost all the device memory. Also, the total_memory_free_blocks should oscillate around a value, instead of keep increasing over time.

Screenshots Some plots comparing memory usage and number of free blocks over time of Wifi and Ethernet. image As can be seen, the number of free blocks, i.e. memory fragmentation, is much higher on WiFi than on Ethernet. Also, WiFi increases over time the total allocated memory, on the other hand, Ethernet keeps it stable.

Meadow (please complete the following information as best as you can):

Board Information 
    Model: F7Micro
    Hardware version: F7CoreComputeV2

Hardware Information 
    Processor type: STM32F777IIK6
    ID: 2E-00-25-00-12-50-33-53-32-34-31-20
    Serial number: 2056346A5333
    Coprocessor type: ESP32
    MAC Address - 
        WiFi: 4C:75:25:D5:28:34

Firmware Versions 
    OS: 1.14.0.0
    Mono: 1.14.0.0
    Coprocessor: 1.14.0.0
    Protocol: 7

Additional context

After two week of running Cultivar sending metrics every minute this is the device metrics info:

{
"health.total_allocated_memory": 26337680,
"health.total_free_memory": 4120176,
"health.cpu_temp_celsius": 43.3443223443223,
"info.os_version": "1.13.1.5",
"info.coprocessor_os_version": "1.12.2.0",
"health.largest_memory_free_block": 3002608,
"health.disk_space_used": 14150883,
"up_time": "14.01:11:17.2190000",
"health.total_memory_arena": 30457856,
"health.memory_used": 1650688,
"health.total_memory_free_blocks": 10273
}

{
"TemperatureCelsius": 34.95703125,
"HumidityPercent": 34.7725743588918,
"SoilMoistureDouble": 2.32686110272057,
"LogId": 19499
}

After 24 days of running, this is the device info:

{
"health.total_allocated_memory": 29057344,
"health.total_free_memory": 1400512,
"health.cpu_temp_celsius": 43.3443223443223,
"info.os_version": "1.13.1.5",
"info.coprocessor_os_version": "1.12.2.0",
"health.largest_memory_free_block": 266128,
"health.disk_space_used": 14150883,
"up_time": "24.05:21:54.3500000",
"health.total_memory_arena": 30457856,
"health.memory_used": 1866464,
"health.total_memory_free_blocks": 14345
}

{
"TemperatureCelsius": 35.34765625,
"HumidityPercent": 38.4992414564003,
"SoilMoistureDouble": 2.32792670385756,
"LogId": 33620
}

As can be seen, the total_free_memory decreased over time due to the leak, as well as the largest_memory_free_block due to the fragmentation.