waterloo-rocketry / cansw_processor_airbrakes

Processor board was developed for the 2023-2024 design cycle to perform state estimation and control tasks for Borealis's airbrakes system
2 stars 0 forks source link

SD card logging is slow #40

Closed Joe-Joe-Joe-Joe closed 2 months ago

Joe-Joe-Joe-Joe commented 3 months ago

The logging API gets bogged down pretty quick to the point it can't keep up with demand. Upping the clock speed of the SDMMC peripheral may fix this, so this should be increased and then log.c load tested (I would just run the debugger and break at the spot in logGeneric where we check to see if the buffer is already full)

celery6 commented 3 months ago

the one time i tested it was writing 1 buffer (8192 byte) per 10 ms. 1 buffer is 32 fullize msgs, so this is like 3khz msgs/sec which shiuldnt be bogged down already? is the math wrong or smth else issue

Joe-Joe-Joe-Joe commented 3 months ago

This is with 25 Mhz SDMMC clock and trying to write 1 log message every 1 tick image

This is with 100 MHz image

Not exactly sure where the issue lies but its bottlenecking at almost exactly 10ms between messages

celery6 commented 3 months ago

ill look at this issue more this tmr

celery6 commented 3 months ago

this might be being bottlenecked at logGeneric() which takes between 1 to 6 ticks depending on msg length.. and only 1 logGeneric can run at once so thats like 1msg per 5 ticks which is way less than SD card write speed of ~3.2 msgs per 1 tick

also because tasks have to wait in line for the logGeneric mutex which is currently up to 50 ticks so uhhh. But that shouldnt be a bottleneck for populating the log buffer

celery6 commented 3 months ago

assume that 1 logGeneric takes 3 ms to complete (not including waiting in line), that's like 3.3 full size msgs per 10 ticks which doesnt seem ideal :skull-emoticon (full size is 256 chars including everything)

Joe-Joe-Joe-Joe commented 2 months ago

this has become a bit of a rabbit hole and this issue is no longer actionable. closing.

Joe-Joe-Joe-Joe commented 2 months ago

I missed that this was covered in #65 Make sure to reference issues in your PR :bulbaowo: