search

GPSBabel / gpsbabel

GPSBabel: convert, manipulate, and transfer data from GPS programs or GPS receivers. Open Source and supported on MacOS, Windows, Linux, and more. Pointy clicky GUI or a command line version...
https://www.gpsbabel.org
GNU General Public License v2.0
470 stars 125 forks source link

simplify formatting of date/time in garmin_txt #1205

Open tsteven4 opened 10 months ago

tsteven4 commented 10 months ago

This is a user visible change.

This is a test of a solution to #1113.

Currently garmin_txt readers and writers use a homegrown "human" date and time format. This is converted to a strptime/strftime format in util.cc by convert_human_date_format, convert_human_time_format.

The garmin_txt reader then converts the strptime/strftime format to a QDateTime format, which is remarkably similar to what we started with! The garmin_txt writer uses strftime.

This PR changes the format for date and time to that of QDateTime, eliminating all the conversions and the use of strftime in garmin_txt.

There are some potential issues with expanding this for general use and thus eliminating strptime/strftime. 1) XcsvFormat::sscanftime counts on strptime not setting fields that aren't used in the format. To do this with Qt we have to scan the format string. Scanning the Qt format string requires processing of quoted text, but that isn't too difficult. 2) QDateTime::fromString(const QString &string, const QString &format) has limited support for fractional seconds. If a decimal point is in the format it is required. ".z" expects 1-3 digits. ".zzz" expects exactly three digits. Note strptime/strpftime didn't support fractional seconds at all. 3) The entire format string must match, partial matches are not supported.

As an aside, the txt/csv format currently used by garmin is incompatible with our implementation, which works with a historical garmin txt/csv format. I suspect our garmin_txt format is mostly useful as a playground for development and to get test coverage of certain features such as garmin_fs category support and init files.

tsteven4 commented 9 months ago

@robertlipe before I explore expanding this idea to replace strptime/strftime I'd like to get your opinion on this approach.

tsteven4 commented 9 months ago

2. Fractional seconds in C89 was one reason we landed up in Qt land. Is the issue that it misparses > 3 digits? so hh:mm:ss.1234 doesn't get that .4Ms? I don't think that makes me sad. Milliseconds is a very reasonable bottom floor when parsing ASCII input, IMO.

No, the issue is QDateTime::fromString returns an invalid date time.

"0" "s" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime)
"0." "s" QDateTime(Invalid)
"0.0" "s" QDateTime(Invalid)
"0.00" "s" QDateTime(Invalid)
"0.000" "s" QDateTime(Invalid)
"0.0000" "s" QDateTime(Invalid)
"10" "s" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10." "s" QDateTime(Invalid)
"10.0" "s" QDateTime(Invalid)
"10.00" "s" QDateTime(Invalid)
"10.000" "s" QDateTime(Invalid)
"10.0000" "s" QDateTime(Invalid)
"0" "s.z" QDateTime(Invalid)
"0." "s.z" QDateTime(Invalid)
"0.0" "s.z" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime)
"0.00" "s.z" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime)
"0.000" "s.z" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime)
"0.0000" "s.z" QDateTime(Invalid)
"10" "s.z" QDateTime(Invalid)
"10." "s.z" QDateTime(Invalid)
"10.0" "s.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.00" "s.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.000" "s.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.0000" "s.z" QDateTime(Invalid)
"0" "s.zzz" QDateTime(Invalid)
"0." "s.zzz" QDateTime(Invalid)
"0.0" "s.zzz" QDateTime(Invalid)
"0.00" "s.zzz" QDateTime(Invalid)
"0.000" "s.zzz" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime)
"0.0000" "s.zzz" QDateTime(Invalid)
"10" "s.zzz" QDateTime(Invalid)
"10." "s.zzz" QDateTime(Invalid)
"10.0" "s.zzz" QDateTime(Invalid)
"10.00" "s.zzz" QDateTime(Invalid)
"10.000" "s.zzz" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.0000" "s.zzz" QDateTime(Invalid)
"0" "ss" QDateTime(Invalid)
"0." "ss" QDateTime(Invalid)
"0.0" "ss" QDateTime(Invalid)
"0.00" "ss" QDateTime(Invalid)
"0.000" "ss" QDateTime(Invalid)
"0.0000" "ss" QDateTime(Invalid)
"10" "ss" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10." "ss" QDateTime(Invalid)
"10.0" "ss" QDateTime(Invalid)
"10.00" "ss" QDateTime(Invalid)
"10.000" "ss" QDateTime(Invalid)
"10.0000" "ss" QDateTime(Invalid)
"0" "ss.z" QDateTime(Invalid)
"0." "ss.z" QDateTime(Invalid)
"0.0" "ss.z" QDateTime(Invalid)
"0.00" "ss.z" QDateTime(Invalid)
"0.000" "ss.z" QDateTime(Invalid)
"0.0000" "ss.z" QDateTime(Invalid)
"10" "ss.z" QDateTime(Invalid)
"10." "ss.z" QDateTime(Invalid)
"10.0" "ss.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.00" "ss.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.000" "ss.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.0000" "ss.z" QDateTime(Invalid)
"0" "ss.zzz" QDateTime(Invalid)
"0." "ss.zzz" QDateTime(Invalid)
"0.0" "ss.zzz" QDateTime(Invalid)
"0.00" "ss.zzz" QDateTime(Invalid)
"0.000" "ss.zzz" QDateTime(Invalid)
"0.0000" "ss.zzz" QDateTime(Invalid)
"10" "ss.zzz" QDateTime(Invalid)
"10." "ss.zzz" QDateTime(Invalid)
"10.0" "ss.zzz" QDateTime(Invalid)
"10.00" "ss.zzz" QDateTime(Invalid)
"10.000" "ss.zzz" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime)
"10.0000" "ss.zzz" QDateTime(Invalid)
GPSBabelDeveloper commented 9 months ago

Gross. So we suppressmtrsiling 0., anything after 3 spaces and reparse. With s.z on failure?

Is it. More amenable to things that look more like times and have an hours and minutes component?

On Thu, Nov 2, 2023, 8:41 AM tsteven4 @.***> wrote:

  1. Fractional seconds in C89 was one reason we landed up in Qt land. Is the issue that it misparses > 3 digits? so hh:mm:ss.1234 doesn't get that .4Ms? I don't think that makes me sad. Milliseconds is a very reasonable bottom floor when parsing ASCII input, IMO.

No, the issue is QDateTime::fromString returns an invalid date time.

"0" "s" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime) "0." "s" QDateTime(Invalid) "0.0" "s" QDateTime(Invalid) "0.00" "s" QDateTime(Invalid) "0.000" "s" QDateTime(Invalid) "0.0000" "s" QDateTime(Invalid) "10" "s" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10." "s" QDateTime(Invalid) "10.0" "s" QDateTime(Invalid) "10.00" "s" QDateTime(Invalid) "10.000" "s" QDateTime(Invalid) "10.0000" "s" QDateTime(Invalid) "0" "s.z" QDateTime(Invalid) "0." "s.z" QDateTime(Invalid) "0.0" "s.z" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime) "0.00" "s.z" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime) "0.000" "s.z" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime) "0.0000" "s.z" QDateTime(Invalid) "10" "s.z" QDateTime(Invalid) "10." "s.z" QDateTime(Invalid) "10.0" "s.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.00" "s.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.000" "s.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.0000" "s.z" QDateTime(Invalid) "0" "s.zzz" QDateTime(Invalid) "0." "s.zzz" QDateTime(Invalid) "0.0" "s.zzz" QDateTime(Invalid) "0.00" "s.zzz" QDateTime(Invalid) "0.000" "s.zzz" QDateTime(1900-01-01 00:00:00.000 MST Qt::LocalTime) "0.0000" "s.zzz" QDateTime(Invalid) "10" "s.zzz" QDateTime(Invalid) "10." "s.zzz" QDateTime(Invalid) "10.0" "s.zzz" QDateTime(Invalid) "10.00" "s.zzz" QDateTime(Invalid) "10.000" "s.zzz" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.0000" "s.zzz" QDateTime(Invalid) "0" "ss" QDateTime(Invalid) "0." "ss" QDateTime(Invalid) "0.0" "ss" QDateTime(Invalid) "0.00" "ss" QDateTime(Invalid) "0.000" "ss" QDateTime(Invalid) "0.0000" "ss" QDateTime(Invalid) "10" "ss" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10." "ss" QDateTime(Invalid) "10.0" "ss" QDateTime(Invalid) "10.00" "ss" QDateTime(Invalid) "10.000" "ss" QDateTime(Invalid) "10.0000" "ss" QDateTime(Invalid) "0" "ss.z" QDateTime(Invalid) "0." "ss.z" QDateTime(Invalid) "0.0" "ss.z" QDateTime(Invalid) "0.00" "ss.z" QDateTime(Invalid) "0.000" "ss.z" QDateTime(Invalid) "0.0000" "ss.z" QDateTime(Invalid) "10" "ss.z" QDateTime(Invalid) "10." "ss.z" QDateTime(Invalid) "10.0" "ss.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.00" "ss.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.000" "ss.z" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.0000" "ss.z" QDateTime(Invalid) "0" "ss.zzz" QDateTime(Invalid) "0." "ss.zzz" QDateTime(Invalid) "0.0" "ss.zzz" QDateTime(Invalid) "0.00" "ss.zzz" QDateTime(Invalid) "0.000" "ss.zzz" QDateTime(Invalid) "0.0000" "ss.zzz" QDateTime(Invalid) "10" "ss.zzz" QDateTime(Invalid) "10." "ss.zzz" QDateTime(Invalid) "10.0" "ss.zzz" QDateTime(Invalid) "10.00" "ss.zzz" QDateTime(Invalid) "10.000" "ss.zzz" QDateTime(1900-01-01 00:00:10.000 MST Qt::LocalTime) "10.0000" "ss.zzz" QDateTime(Invalid)

— Reply to this email directly, view it on GitHub https://github.com/GPSBabel/gpsbabel/pull/1205#issuecomment-1790757536, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC3VAD3LISGWQCNWCE3KLWDYCOPIPAVCNFSM6AAAAAA6ZMC2T6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJQG42TONJTGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>