tlog parser

[mnumanuyar]

tlog files are commonly used in mavlink community , and there are already opensource javascript examples to parse them in uavlogviewer repo It would be very usefull for a mavlink library to allow users to parse tlog files, with time info I am not sure what is the correct way to implement this, or whether it should be iplemented.

To parse tlog files, most of the work is done in MavLinkPacketSplitter but user also needs to parse the 8 bytes before the packet start marker, which indicates timestamp

[mnumanuyar]

if this feature is not planned, i could spare my time for development and/or test. but as i said, i am not sure about the correct way to implement this. Or if this is possible someway, an example would be usefull

[padcom]

@mnumanuyar It's a good idea, and not hard to implement. If you want to take it, be my guest. It should be as easy as extracting timestamp to a configuration option and making a new class, TLogPacketSplitter that calls the super constructor.

[mnumanuyar]

sorry, I didnt quite understand where to extract time info. What do you mean by "extracting timestamp to a configuration option"? Currently I parsed time info into log, and I can certainly make it into another class. But how can I make it so that user would be able to parse the splitted packets with MavLinkPacketParser and associate each packet with correct time stamp?

[padcom]

@mnumanuyar I looked at your proposed implementation. I'll put together something that works later Today. Could you attach to this ticket a tlog file? I don't have a device at this moment that could produce such file.

[mnumanuyar]

Thank you very much :D I used UAVLogViewers sample log for testing

[mnumanuyar]

Any progress? or anything I can help with? If there is a relevant example I can try to mimic it and test it. Or if there is a relevant nodejs concept, I can read the docs and try to apply it

[padcom]

Not yet, but I am working on it.

czw., 19 maj 2022 o 07:49 numan @.***> napisał(a):

Any progress? or anything I can help with? If there is a relevant example I can try to mimic it and test it. Or if there is a relevant nodejs concept, I can read the docs and try to apply it

— Reply to this email directly, view it on GitHub https://github.com/ArduPilot/node-mavlink/issues/10#issuecomment-1131237343, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEHICQBLY3AFVIJPTCR5QDVKXI6PANCNFSM5VQ32YXA . You are receiving this because you commented.Message ID: @.***>

[padcom]

@mnumanuyar

I have an idea that might just work. I'll implement it next week. It goes like this:

• remember last 4/8 bytes read from the stream (let's call it timestampBuffer)
• proceed as normal with splitting (this should find the start of the packet)
• when emitting a buffer with packet add metadata in the form of { timestamp: timestampBuffer.readWhateverTimestamp(0) }
• in the packet parser put that timestamp field in the header if a configuration option (let's say readPacketTimestamp) is switched on

I'll have to see about passing on metadata (maybe I need to switch from passing buffers to passing objects all together)... will see.

[padcom]

@mnumanuyar

Ok, I couldn't let go. Please try version 1.2.0. It has, in the timestamp in packet.header.timestamp (or null if it was not present). No extra configuration needed.

[mnumanuyar]

thank you very much, I will try as soon as I can :D

[mnumanuyar]

Unfortunatly I got an error on my tests:

TypeError: Cannot read properties of undefined (reading '76')
    at MavLinkPacketSplitter.validatePacket (D:\GitRepos\yki\node_modules\node-mavlink\dist\lib\mavlink.js:1:10582)
    at MavLinkPacketSplitter._transform (D:\GitRepos\yki\node_modules\node-mavlink\dist\lib\mavlink.js:1:9569)

I have not yet able to look into it further, it might be a problem on my end aswell (faulty tlog file etc) On that note, i noticed js files of node-mavlink are jumbled a bit (minified? it is not human readable), which makes it harder to debug for me (it is hard to fallow error messages and try to decode what line they are really from). So i probably have to learn how to pretty print those first

[padcom]

Can you try the parse-tlog-file.ts example and see if that works?

[padcom]

Better yet, why don't you send me the file you're trying to parse (if it's any different than the one you linked to this post before). That way I'll be able to reproduce the case, fix the problem and everyone will be happy :)

[mnumanuyar]

ok, it worked when I downloaded the repo and run parse-tlog-file.ts even after I changed the path so it showed my tlog file.

maybe there is a config issue in my repo. (which has node-mavlink@1.2.1 as a nodemodule installed via npm)

I tihnk we can close this issue. Thank you very much for your help :D