MSC (usb mass storage) mode corrupting blackbox logs

hughsaunders commented 5 months ago

Current Behavior

I downloaded blackbox logs via msc and couldn't open them. I then took the sdcard out of the flight controller and inserted it directly into the laptop and copied the same logs. The SHA signatures are different between the two sets of files, also blackbox_decode can read the files from the sd copy but not from the msc copy.

I tried the msc copy multiple times and got the same SHA each time, it wasn't a one off error.

SHAS:

hMBPpersonalWired:black box logs hughsaunders$ shasum */*.TXT
c1606a5549259c6a96a56c6a9ef5520bf389927a  msc copy/LOG00001.TXT
51523a8ec33ed9c03b0f2ecfe469ec4924cab49b  msc copy/LOG00002.TXT
3dd26091de20abc46d58029789453a120f7c23c8  sd copy/LOG00001.TXT
0d959f5d831db6783cf83e668ef7cbfb7878a73b  sd copy/LOG00002.TXT

Attempt to decode msc copy files:

hMBPpersonalWired:black box logs hughsaunders$ blackbox_decode msc\ copy/LOG00001.TXT 
Couldn't find the header of a flight log in the file 'msc copy/LOG00001.TXT', is this the right kind of file?

Attempt to decode SD copy files:

hMBPpersonalWired:black box logs hughsaunders$ blackbox_decode sd\ copy/LOG00001.TXT 
Decoding log 'sd copy/LOG00001.TXT' to 'sd copy/LOG00001.01.csv'...

Log 1 of 1, start 04:51.512, end 20:57.935, duration 16:06.422

Statistics
Looptime           1005 avg            3.3 std dev (0.3%)
I frames   30016  117.1 bytes avg  3514012 bytes total
P frames   30007   77.4 bytes avg  2321341 bytes total
H frames     236   10.0 bytes avg     2360 bytes total
G frames    9760   24.5 bytes avg   239419 bytes total
E frames       1    7.0 bytes avg        7 bytes total
S frames    3588   45.8 bytes avg   164497 bytes total
Frames     60023   97.2 bytes avg  5835353 bytes total
Data rate   62Hz   6463 bytes/s      64700 baud

25 frames failed to decode, rendering 9 loop iterations unreadable. 9 iterations are missing in total (9ms, 0.00%)
900465 loop iterations weren't logged because of your blackbox_rate settings (906019ms, 93.75%)

Generated by INAV blackbox_decode 7.1.0 16e6283

Steps to Reproduce

Configure black box logs and go flying
Connect FC via usb, open configurator and on CLI tab enter msc
FC enters usb mas storage mode
Copy BBL files via OS standard method
Attempt to decode BBL via blackbox_decode and find them corrupted.

Expected behavior

Copying blackbox logs via MSC works the same as removing the sdcard and copying the files manually.

Additional context

CLI dump: https://pastebin.com/UercZGc2 Host OS: MacOS 12.7.4

I notice from this document, that msc copy and sd copy should be equivalent: https://github.com/iNavFlight/inav/blob/master/docs/USB_Mass_Storage_(MSC)_mode.md

FC Board name and vendor: Atomrc F405 Navi

INAV version string:

# INAV/ATOMRCF405NAVI 7.1.0 Mar 27 2024 / 12:49:53 (59a6ee61)
# GCC-10.3.1 20210824 (release)

hughsaunders commented 4 months ago

This turned out to be a problem with my sd card, re-formatted and it now works fine with the msc command.

lienmeat commented 3 months ago

I was having this exact same issue with a 128gb sd card that was indeed formatted with MBR FAT32. If you plug the SD card into a computer you can read the logs that way, but it will not work with MSC (usb cable) with a 128gb partition. However, if you re-partition the sd card to have only a 4gb partition, MSC works perfectly to read the logs. I did not find this documented anywhere, I just had a hunch that it might not work due large sd cards due to an old forum thread saying iNav didn't support writing past 4gb on large sd cards (I don't even know if that is true, but MSC definitely doesn't like very large partitions).

I made a video explaining the problem and showing how to fix it on MacOs, but I had the same issue with MSC on Windows and Linux. youtube vid inav msc corrupted blackbox data

If this is a known limitation, please make it easier to find (perhaps on the inav configurator UI CLI page or msc command documentation). Without knowing more about MSC and inav, I would consider it a bug, but it may not be.

iNavFlight / inav