wader / fq

jq for binary formats - tool, language and decoders for working with binary and text formats
Other
9.57k stars 219 forks source link

fit: Added support for ANT+ FIT format (used by Garmin devices) #863

Closed mlofjard closed 4 months ago

wader commented 5 months ago

Nice! will have a look tomorrow, but some quick notes:

Some tests and test files would be nice. Any idea if there are any good ones around under liberal license? alternatively is there some good way to produce own test files?

Possible to include the scripts used to generate the tables, even if messy? can also have a README.md under the fit directory etc with instructions how to regenerate etc

mlofjard commented 5 months ago

Yes, I will check the license on the SDK example files, and if they are not liberal enough I will throw in some of my own files. I'll also do some linting/cleanup and put my nodeJS script in the dev-folder, along with some markdown documentation in the fit-folder as suggested.

wader commented 5 months ago

Yes, I will check the license on the SDK example files, and if they are not liberal enough I will throw in some of my own files.

👍

I'll also do some linting/cleanup and put my nodeJS script in the dev-folder, along with some markdown documentation in the fit-folder as suggested.

I usually try keep all format specific things in format/<name> and if test related like tests, test files or script to generate things keep them in testdata.

btw to add authorship, references, known limitations, help etc you can add a fit.md and include it, ex: https://github.com/wader/fq/blob/master/format/bson/bson.md https://github.com/wader/fq/blob/master/format/bson/bson.go#L14-L27 that will then be used for cli help fq -h fit and to generated usage documentation (make doc)

wader commented 5 months ago

Output looks really good, impressive work 👍 hope i get more time to review tomorrow

wader commented 5 months ago

Should have been reviewing the code but ended up play around with it! try this:

go run . -r '{kml: {"@xmlns": "http://www.opengis.net/kml/2.2", Document: {Placemark: {LineString: {coordinates: [.dataRecords[].dataMessage | select(.position_lat and .position_long) | "\(.position_long),\(.position_lat)"] | join("\n")}}}}} | to_xml'  format/fit/testdata/activity.fit > test.kml
mlofjard commented 5 months ago

Should have been reviewing the code but ended up play around with it! try this:

go run . -r '{kml: {"@xmlns": "http://www.opengis.net/kml/2.2", Document: {Placemark: {LineString: {coordinates: [.dataRecords[].dataMessage | select(.position_lat and .position_long) | "\(.position_long),\(.position_lat)"] | join("\n")}}}}} | to_xml'  format/fit/testdata/activity.fit > test.kml

Yup, that is the route for STHLM Bike 2023 =)

mlofjard commented 5 months ago

Some minor comments. Do you have something left you want to change or add before merge?

I have some stuff left that I know of, e.g (dynamic/subfields) and accumulated timestamp values from compressed messages. But the subfields are a bigger task, and I have no files with compressed message headers to test on. So I think they'll both have to wait for future versions.

wader commented 4 months ago

Merge once CI is green?

wader commented 4 months ago

🥳 Thanks a lot and hope for more PRs in the future!

btw i know a colleague of mine that is super excited about fit support, ping @KristianKarl

mlofjard commented 4 months ago

🥳 Thanks a lot and hope for more PRs in the future!

btw i know a colleague of mine that is super excited about fit support, ping @KristianKarl

I have already started on my next format: a nes rom decoder, with a to_asm function for decompiling to 6502 assembly code. =)

wader commented 4 months ago

Oh nice, it's addictive 😄 to_asm as in return a string with 6502 assembly? interesting 🤔

I have played around with the idea of having opcode decoders but got a bit stuck, didn't feel very neat, maybe could be useful? ex draft PR https://github.com/wader/fq/pull/215

wader commented 4 months ago

BTW i'm think of will do a new release later today or tomorrow, has been a while. Just do a PR if you want to include some smaller fixes.

And if you happen to have some nice examples to show up the fit decoder drop them here and will include in the release notes! was thinking about including covert to KML maybe?

KristianKarl commented 4 months ago

Most awesome PR! Great job both! :clap:

I already used fq and it's new fit decoder to create a summary of each of my 1300+ fit files in 1 json file. This file is super easy to query using jq to get my longest, fastest etc ride, or to get the total distance for a specific bike. Very handy, I'll do a write-up shortly how to do that,

wader commented 4 months ago

🥳 Nice! i've thought about splitting up https://github.com/wader/fq/blob/master/doc/formats.md somehow to have a page per format, maybe even use the github wiki? then write ups like that could be included?