Log files naming

[lvale]

For those that deal with several vehicles, the current naming scheme of the logs requires very cumbersome methods to organize.

I would like to suggest to @peterbarker something like this:

YYMMDDHHMMXX.bin

where XX can be AP (ArduPlane) AC ArduCopter, AH ArduHeli, AR ARduRover, AS ArduSub, etc...

and yymmddhhmm would be date/time of the start of the log.

If we can have longer filenames we could get some separators to make it easier to read.

and would become

YYMMDDHHMMXXZZZ.bin

where ZZZ would be Mavlink ID of the vehicle, which would be great when you are using swarms :)

also related with https://github.com/ArduPilot/ardupilot/issues/5771

[Pedals2Paddles]

Yes please x1000!!

Does the autopilot get it's date/time from the GPS, and maintain it once it is initially acquired? Under what circumstances does the autopilot to lose it's date/time?

[peterbarker]

On Wed, 1 Nov 2017, Matt wrote:

Does the autopilot get it's date/time from the GPS, and maintain it once it

Yes.

is initially acquired? Under what circumstances does the autopilot to lose it's date/time?

It will never lose its date and time, but it will often never get it - which is why you end up with logs with a timestamp in the 2000's.

[WickedShell]

It will never lose its date and time, but it will often never get it - which is why you end up with logs with a timestamp in the 2000's.

1980-01-05 17:00 + jitter (up to 48 minutes worth) is the one I've been seeing recently. The 1999 stuff I haven't seen within the last month (although I was seeing it before that)

[OlivierJB]

+1 on this one, especially the Mav ID part.

[lvale]

@tridge I believe you also had some ideas about this ? @rmackay9 any ideas too ?

[beachbum1973]

There are a couple of improvements I think... depending on how you decide to download the log files...

1. Download via Mavlink Having a common naming convention would be good.. it seems as if there is a different just between ArduCopter & ArduPlane.. one uses YYMMDD while the other uses MMDDYY... (is this in mission planner? or in the arduplane/arducopter code????)

It also appears to be a difference the time (no so much date) shown in the download screen vs the filename once downloaded.. can be very confusing... Or am I missing something here???

Is the file name created by mission planner (based on info in the .bin) after the download via mavlink, if so, is this issue more related to the mission planner vs the ardupilot code...

Note: I use mission planner as my GCS software...

W.r.t. the ardupilot code... does this relate to the naming on the SD card.. and hence...

1. Removal and direct copy of SD card. From my knowledge, the log files on the SD card are stored in a xxx.bin where xxx is just a number... It would be good to be able have a feature (probably on mission planner) that would allow you to take the xxx.bin and copy/rename to have the same naming convention that has been suggested above or agreed too... that is, takes the xxx.bin and produces the YYMMDDHHMMXXZZZ.bin...

So.. if the file naming is created by Mission Planner vs in the copter/plane/ardupilot code can a common file name convention be decided that could be longer, use delimiters, relate to the vehicle (copter, plane, sub, etc) and include the mav ID as well... some thought w.r.t. useful naming vs sorting on file manager would be good too... date before time.. etc...

@tridge , @rmackay9 @meee1 @lthall ... Are you able to provide any comment or corrections to my discussions?

[lvale]

@beachbum1973 I think it would be best to separate what the GCS does from what / how the flight stack handles the log files. At least for now.

[beachbum1973]

By flight stack, are you referring to the code that runs on the hardware? if so, my understanding is that the log files are just numbered .bin files on the SD card... e.g. "3.bin", where the "lastlog.txt"file is used to store the last number used...

Is this what you are suggesting gets modified to utilise the longer naming convention? if so, would there be a problem with lack of GPS info (date, time etc) for the code to correctly determine the file name?

[lvale]

One possible option when GPS or other reliable time source is not available would be to increment on the lastlog.txt with a bogus hour example lastlog.txt - 1711101505AC001.bin

next operation would start and operate without reliable time source so next flight logs would be

1711102401AC001.bin

1711102402AC001.bin

This fake 24+ hour would be easily spotted and users would know that if the hour was greater than 24 there wasn't a reliable time source.

Corner case

Empty lastlog.txt and no reliable time source

First log would have the unix initial date and hour would follow the same pattern

@peterbarker Would this be doable ?

[peterbarker]

On Sat, 11 Nov 2017, Luis Vale Gonçalves wrote:

@peterbarker Would this be doable ?

I don't see any reason why not - but I'd find it rather confusing!

[OXINARF]

@lvale What happens when you later do a flight with GPS and the filename is supposed to be 1711101800AC001.bin? Do you ignore the current time because it was unknown before or you use it and make ordering useless?

This proposal has a great intention but I don't see a way to do it without making it more confusing.

[lvale]

@OXINARF I fail to see your point. If there is GPS no 2 logs will have the same name If the last flight had a name that was generated by GPS and the next flight has no GPS, the fake 24+ hour takes care of that

1711101800AC001.bin GPS available 1711101805AC001.bin GPS available 1711102401AC001.bin No GPS 1711102402AC001.bin No GPS 1711102403AC001.bin No GPS 1711101806AC001.bin GPS available

I'm sure that a real GPS will never give you a 24+ hour timestamp

[OXINARF]

@lvale Except the order of those files that any file explorer will show you are:

1711101800AC001.bin GPS available 1711101805AC001.bin GPS available 1711101806AC001.bin GPS available 1711102401AC001.bin No GPS 1711102402AC001.bin No GPS 1711102403AC001.bin No GPS

Now you have no idea where those last three files fit between the first three files.

[Pedals2Paddles]

Some suggestions that would help with sorting and readability:

• Put the sequence # first, so they are always in order whether there was a valid date/time or not.
• Use the 4 digit year to avoid ambiguity between the year 2017 and the 17th day of a month.
• Underscores help readability between segments of information
• Don't date/time stamp it if a legit date/time isn't available. No need for fake information.

AC001_2017-11-17_1857.bin AC002_2017-11-17_1922.bin AC003_2017-11-180517.bin AC004.bin (example where no date/time is available)_ AC005_2017-11-18_0722.bin

Is there a filename length constraint? That's 25 characters with the extension.

[OXINARF]

@Pedals2Paddles Interesting proposal, I like that one.

[lvale]

@Pedals2Paddles It's missing the MavlinkID of the vehicle on purpose ?

[OlivierJB]

+1 on @Pedals2Paddles enhancement to Luis proposal. But yes, with Mavlink ID also.

[Pedals2Paddles]

Hadn't thought of that. I would put the mavlink ID between AC and the sequence #. So when you're sorting files, the vehicles stay grouped together in flight order.

[vehicle type abbreviation][Mavlink ID][sequence #][date][time].bin

AC_999_001_2017-11-17_1857.bin AC_999_002_2017-11-17_1922.bin AC_999_003_2017-11-18_0517.bin AC_999_004.bin (example where no date/time is available) AC_999_005_2017-11-18_0722.bin

It's lengthy, but extremely informative and easy to follow.

[lvale]

I think that adding all the dashes makes it longer than required, but it's a matter of opinion... :)

AC999001_201711171857.bin AC999002_201711171922.bin AC999003_201711180517.bin AC999004.bin (example where no date/time is available) AC999005_201711180722.bin

[amilcarlucas]

What is the status of this ?

[Naterater]

Bump on this. I'm also running into issues.

[peterbarker]

tridge has concerns with long filenames. 2 fat entries are used, leading to atomicity issues. It is unclear what impact this might have on reliability of the SD card storage.

[lvale]

sd cards are disposable, and so cheap that reliability in the tens of years is of no big concern