JosephHewitt / wardriver_rev3

A portable ESP32-based WiFi/Bluetooth scanner for Wigle.net.
https://wardriver.uk
GNU General Public License v3.0
251 stars 25 forks source link

Issues with historical captures when resetting boot counter #154

Closed JosephHewitt closed 3 months ago

JosephHewitt commented 4 months ago

When bc is reset (due to a factory reset), new files will be generated with low IDs again meaning the filenames can collide with previous capture files.

Capture files are currently opened with FILE_APPEND meaning new data can be appended to old captures. This likely won't result in any lost capture data at all, but will cause duplicate data when uploading to WiGLE and could cause user confusion.

I can think of a few ways to fix this, but I think this would be the best approach:

On first boot (or after a factory reset), iterate the current capture files and find the highest ID. This can then be used to set bc in the NVS so the captures will continue exactly as they did before the reset. This does mean the factory reset becomes slightly less 'hard' than previously implemented, but the documentation can also be updated to encourage users to delete all of their capture files before resetting the device if they are experiencing issues; this will cause the bc value to be fully reset along with everything else.

pejacoby commented 4 months ago

One additional issue identified -- as the 'new' file numbers overlap with uploads from a long time ago, Wigle is returning status to the auto-upload process indicating it already has the file, so no upload occurs. Screenshot attached of two runs today.

The first run was an hour walk with a file size of 310kb. Obviously there are more than "0 total" APs in this file -- no upload was attempted after a reboot.

The second file is from a quick reboot where I let everything fire up and start scanning, then shut down. Here too the file size and WiFi total count don't make sense -- 2kb file yes, but containing 3157 APs is unlikely. Again no upload attempted here on the reboot.

Previous files since the factory reset auto-uploaded but I'm guessing that is because there was no matching filename found in the call to Wigle.

Let me know if you need any other info. I was able to upload the larger file via the Upload link in the interface and Wigle is processing it.

Wispy-Uploads Ignored-Dupe file numbers

JosephHewitt commented 3 months ago

Sorry for not responding sooner, @pejacoby

This appears to be a separate issue. If I'm understanding correctly, it seems the wardriver is not correctly matching local files with files which WiGLE has. This is meant to be achieved with 3 variables: The wardriver hardware ID, the bootcount value, and the filesize. These matching by coincidence is unlikely so it seems there is a bug in the implementation.

I'm working on a PR which will close this Github issue because it fixes the original problem. I am still interested in investigating/fixing your issue however. Please could you open a separate issue for it? Otherwise it makes it hard for me to keep track of the issues. Thanks! :)

pejacoby commented 3 months ago

I'll open a new issue for the second situation - thanks for the fix on this one!