lexer should know frame lengths

lkarsten / fdxread

Nexus FDX decoder for Garmin GND10

GNU General Public License v2.0

15 stars 4 forks source link

lexer should know frame lengths #8

Closed lkarsten closed 7 years ago

lkarsten commented 7 years ago

Currently the lexer looks for a single 0x81 and produces messages based on that. This is a bit crude.

$ ./fdxread --format raw dumps/onsdagsregatta-2016-08-17.dump 2>&1 1>/dev/null| head -10
WARNING:root:mtype=0x010405: Incorrect length 9 (got 8.0) body: 7f01708f81
WARNING:root:mtype=0x010405: Incorrect length 9 (got 6.0) body: 370181

Looking up the hex sequence of the first warning, finding this line in onsdagsregatta-2016-08-17.dump: 1471447636.547 42 01 04 05 7f 01 70 8f 81 81 11 02 13 [.. cut]

The 0x010405 message's last octet is 0x81, which is mistaken for a message delimiter. The next message is just the 0x81 (1 octet) and is skipped.

Solution here is to have a smarter fdx reader that recognises the 3 byte frame type and then consumes the correct number of octets.

ygageot commented 7 years ago

Did you found a documentation about the meaning of 3 bytes frame ?

lkarsten commented 7 years ago

There is no public documentation on FDX, as far as I know. There are mentions online of a developer document from Nexus AB, but I haven't been able to find it. If anyone has a copy, send it to me!

I've written a smarter lexer that knows about frame lengths, but it came with a major reorganization of decode.py so I decided to take another pass at it before commiting it.

ygageot commented 7 years ago

To understand the second byte with frame length is an large advance.

We have to understand the first and the last byte ...

My impressions are

the 0x81 byte is not at the end of the frame but is at the beginning, it permit the receiving software to decode the 3 bytes just after and permit easy synchronization when the receiver read on the fly.
somewhere is the Unit ID of the equipment (32 max equipments=5 bits) The WSI box has always ID=2 and NX2 server has ID=0; the other equipments have usually IDs 16,17,...

lkarsten commented 7 years ago

I forgot to update this issue when I did 13faaa45.

It implements a better tokenizer that uses the frame length in the header.

I agree that 0x81 should be looked at as the start byte and not the end byte. It makes more sense now that we know that the header has a length field.

Most of my data comes from the GND10 box, so getting a grasp on what may be the device id isn't easy since it generates all of them. If anyone has more data dumps from actual NX2 systems, I think that would help.

Closing this as the main issue has been solved.

ygageot commented 7 years ago

Do you known the Unit ID of your GND10 ?