grabserial as HEX dump data logger

[anarh6]

Hi there!

It will not be a report of issue, it will be more like question/idea for software. I want to make serial port HEX data logger from Pi4.

I use grabserial software, but it saves and show data only after decoding, but I need to see raw HEX DUMP. I need to see raw data, because its communication commands. Like command to startup: 0xGG 0xZE 0x22, not a word.

I modified file "grabserial" by deleting all decode of UTF-8, but it still continue to decode it. If i understand correctly, grabserial works with unicode, so it's extremely bad idea to do but still i did it.

Now when i send to serial port 0xCC 0xFC from grabserial I see: [14:25:05.770066 0.000002] ▒

But I want to see and save to LOG: [14:25:05.770066 0.000002] CC FC

Is there any ideas how to get raw HEX data ?

P.S.

I am new in Linux and Raspberry PI. In case if I asked or write something stupid, i'm sorry.

Thanks in advance, Best regards, Ainars

[tbird20d]

I'd be willing to help you with a patch for this. Here are some ideas I have on the topic:

I think you should add a new command-line option for this feature, called '--hex-output'. You can do this by adding the parsing for the option about line 562. Please follow the conventions used for parsing other options. When you see the option, set a flag (maybe called 'hex_output'?) Make sure the flag is pre-initialized to False, in case the option is not seen.

As you can see, at about line 799 in grabserial the received character ('x') is converted into 'out_char', which is the character that will be printed. At line 862, the out_char is written to the stdout, and at line 865, the endecoded character ('x') is put to the log (output file).

On these lines, if the hex_output flag is set, instead of writing out_char or x, write x converted to a hex code. You could do something like: outputfd.write("%0X' % ord(x)). Do it in both places where output is performed.

Please make these changes, and test them, and if when it works to your satisfaction send me a patch or a pull request, and I'll take a look at it. If you're not a programmer at all, let me know, and I'll work on this when I have some free time.

[anarh6]

Thanks for replay!

I am not a programmer at all, I can only trust to you this question.

Thanks for future patch!

[AlphaSierraHotel]

FYI, I hacked up something like this a few months back while I was working on an RS485 interface. @tbird20d, If you want, I can post a gist of it this weekend and follow up here with a link.

[AlphaSierraHotel]

Here is a Gist that illustrates what I did to get grabserial to output data in hex format.

I didn't create a pull-request for this for the following reasons.

1. I don't have a convenient way to test this code on-hand at the moment
2. It's not really a drop-in solution for this issue/feature request.
3. The Gist code itself may provide the functionality to the OP to do what he wants to do, without integrating it into the codebase.
4. I'm of the opinion that it doesn't fit well with the codebase due to the added complexity of non-text-based serial communication protocols.
5. There are other solutions/ways to capture binary serial line communications that are better suited to this functionality and use-case.

The code provided in the Gist was modelled on the serial communication protocol that I was analyzing at the time which used timing to delineate packet data (thus the change made to "sd.timeout" in the code). There were no newlines or start/stop characters in use, which is a bit different from the normal grabserial use-case. It would though be more common-place in the use-cases for viewing or capturing the data stream in hex.

In the end, I wrote a completely different script to capture the data and calculate/compare the CRC bits to validate the packets.

[anarh6]

@AlphaSierraHotel , thnx for youre response. I checked your modification. As written in your modified file: > "Output the hex values of the characters received rather than the characters themselves".

Send in HEX: fc 0b c1 1c Grabserial recived: 0B 1C So it can't display HEX which is not a character.

[AlphaSierraHotel]

Well as I mentioned, it was a bit of a quick hack that wasn't fully tested or utilized.

Here's a starting point for my simplified script for reading binary data from the serial port. Try it and see if that gives you what you need.

Otherwise, I guess we'll have to wait until @tbird20d has a chance to look into adding a proper binary hex dump option to grabserial.

[tbird20d]

OK - I found some time to write something up and test it. It seems to work OK for me, but I don't have wonky codes coming through my serial port. It's in the 'hexoutput' branch. Please give it a try and let me know if it works for you. I'm not super happy with it, as it adds some conditionals on the fastpath in the code. In general, I think this kind of thing is better handled by a post-grab conversion utility. However, let me know if it works for you and does what you'd expect. Note that the output has no line feeds, since those are also converted to hex codes in the output.

Add '--hex-output' to your command line options to activate this output mode (or see the usage help).

If you could describe a little bit more what this will be used for, it would help me determine if this is worth adding to the master branch or not.

[anarh6]

Hi, thnx for replays! Basically my task is to make a datalogger with at least 4 serial ports(with the ability to configure ports) and save all captured data to files in as HEX DUMP. The commands which will be captured do not have any meaning. Command looks like that: fc ea 1c c1 and this data must be captured.

Thanks to @AlphaSierraHotel, I was able to do that, thank you! When I will finish task and check everything, I'll post the script here. Just to stay it in history.

@tbird20d, i check still code does not save or print all captured data for example i send to serial port command in HEX: fc ea 1c c1. Grab serial capture only [13:10:26.734984 0.000003] 1C. So values of fc ea c1 ar not saved.

P.S. @tbird20d Thank you so much for being so responsive.

[AlphaSierraHotel]

Hi Ainars,

Can you test the code from my fork of the hexoutput branch of grabserial and report back with the results?

[anarh6]

Hi @AlphaSierraHotel, I tested it and it works correctly. It prints all data received on serial port in HEX format. Thanks you for your support and ideas!

Ways of improvements: I send the data three times, at different times. But the recording takes place in one line, although the time of receiving the message has changed.

[tbird20d]

I believe all support for the basic feature has been completed. I'm not sure I understand the suggested improvement. Can you please open this on a separate issue? Thanks.

[AlphaSierraHotel]

PR tbird20d/grabserial/pull/53 addressed this suggested improvement. As you say, the feature is complete and it's now in the master branch.