search

PX4 / PX4-Autopilot

PX4 Autopilot Software
https://px4.io
BSD 3-Clause "New" or "Revised" License
8.17k stars 13.36k forks source link

dataman file contains random data after creation #21278

Closed digitalw0lf closed 1 year ago

digitalw0lf commented 1 year ago

Describe the bug

When PX4 creates a dataman file on SD card, contents of this file is not cleared, but contains random data from deleted files on that disk. This is because Nuttx is not fully compliant with Posix standard: using lseek() to expand file size does not erases resulting "gap" in file on FAT32 (yes, FAT32 does not support real sparse files, but according to specs, this "gap" should read as zero bytes anyway). If it happens that this random data looks similar to real dataman records, then PX4 tries to read and process it.

To Reproduce

Steps to reproduce the behavior:

  1. Take new SD card or delete dataman file on existing SD card.
  2. Write large file(s) with non-zero content to SD card and delete them to fill all unused space with garbage. I used a sequence of 0x04000000 that looks like a header of dataman record.
  3. Run PX4 with this SD card.
  4. Check error messages on connection to GCS.
  5. Download dataman file and view it in hex editor. It is filled with garbage from deleted file, except the "compatibility key" at file end.

Expected behavior

dataman file should be filled with zero bytes on creation.

Log Files and Screenshots

Mission read errors Mission transfer failed Hextor

Drone (please complete the following information):

digitalw0lf commented 1 year ago

I've tried to fix this bug by filling entire dataman file with zero's after creation, but encountered another weird behavior: writing of compatibility key failed after that. After some trials and errors, I've settled with strange but working solution: fill entire file except last kilobyte. dm.patch

Igor-Misic commented 1 year ago

Solved here: https://github.com/PX4/PX4-Autopilot/pull/21594