Support for ASCII-safe wire encoding

[siku2]

I'm currently working on a platform which isn't supported by probe-rs yet. I'm also working on that, but as of right now my only way to debug the target is using J-Link's GDB server. The problem with that is that its RTT implementation really doesn't like raw bytes, or to be more specific, interprets them as control commands for itself. For instance, FF followed by anything from 00 to 0F will instruct the client to switch its virtual terminal (another mux mechanism on top of the usual RTT channel) causing subsequent bytes to be routed there instead. This issue previously came up in #558.

I get that it's easy to blame this on the J-Link tools, but I feel like this library could really benefit from an additional encoding to cover such cases. For instance, this point is also brought up in #714:

Often these other tools expect utf-8 encoded data, or don't handle reconnection well, or print data other than the raw bytes to their stdout. Basically, these tools tend to not work well as a source for defmt-print's pipe.

Alternatively, this could be achieved using a custom logger, but that would require a second version of every existing logger (e.g. RTT and UART) and wouldn't be picked up automatically by tools like defmt-print.

As for what this encoding might look like, I'm thinking we could encode the frame content using base64 (or ascii85) and use a space as the end-of-frame indicator.

I'm happy to contribute the implementation but wanted to create this issue first to test the waters.

[hydra]

I wasn't aware of the way that J-Link RTT Viewer virtual terminals worked and wonder if this is also the reason why piping from the RTT Logger would randomly stop?

I implemented this PR for defmt-print as a workaround which seems to work with the J-Link GDB server - https://github.com/knurling-rs/defmt/pull/676.

I use both OpenOCD and J-Link, always with my defmt-print with the TCP option from the above PR and it works well for me currently, but I'd like to see a code example of what causes the problem to manifest on the J-Link GDB server and/or the J-Link RTT Logger and/or the J-Link RTT Viewer. Do you have a small example project/playground/snippet which demonstrates the issue?

[hydra]

For some additional context, the J-Link RTT documentation is here:

https://www.segger.com/products/debug-probes/j-link/technology/about-real-time-transfer/#api-functions

I imagine that the API function 'SEGGER_RTT_SetTerminal()' would be where the 'control commands' are generated in C code and which defmt_rtt accidentally creates in normal use.

[siku2]

I use both OpenOCD and J-Link, always with my defmt-print with the TCP option from the above PR and it works well for me currently

@hydra, first, thank you for your comments. I'm glad to see I'm not the only one dealing with this. This statement is very interesting to me because that's not what happened in my case. The first thing I tried after realizing that JLinkRTTClient only produces scrambled data was to write a simple script which would connect to the J-Link GDB server over telnet and write the output to defmt-print's stdin. This resulted in exactly the same broken output though.

Repeatedly printing an increasing number alone will at some point produce the sequence FF XX:

let mut i = 0;
loop {
  defmt::println!("hello world: {}", i);
  i += 1;
}

I ended up creating this absolutely wild script: https://gist.github.com/siku2/a33be938efb2b9bd54c4d3d7e9c004db which uses the web interface hosted by the GDB server to turn on the RTT function and switching the website to hex output mode, which I'm then parsing back into bytes.

However, thanks to your comment I realized I made a silly mistake. I ended up using a convenient telnet library instead of opening a raw TCP connection, completely forgetting that telnet has the hole 'interpret-as-command' sequence incidentally also started by a 0xFF byte. Now that I think about it that's probably the reason the SEGGER library uses this mechanism in the first place...

---

I imagine that the API function 'SEGGER_RTT_SetTerminal()' would be where the 'control commands' are generated in C code and which defmt_rtt accidentally creates in normal use.

Yes exactly, you can view the source code in the J-Link installation ($INSTALL_DIR/SEGGER/JLink/Samples/RTT/SEGGER_RTT_V784a.zip/RTT/SEGGER_RTT.c) and it basically amounts to this:

/* this is a simplified version compared to the actual function */
int SEGGER_RTT_SetTerminal (unsigned char TerminalId) {
  static unsigned char _aTerminalId[16] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' };

  unsigned char         ac[2];
  ac[0] = 0xFFu;
  ac[1] = _aTerminalId[TerminalId];
  _WriteBlocking(pRing, (const char*)ac, 2u);
  return 0;
}

[hydra]

Yes exactly, you can view the source code in the J-Link installation

That's what I thought, then I +1 this feature request so that DeFmt can work with the Segger debuggers which are generally much better than OpenOCD/STLink.

I'm sure the developers of defmt already know, but If you weren't already aware than you can replace the ST Link firmware with J-Link so if you have a supported Nucleo/Discovery board you can run the J-Link tools that require encoding without having to purchase a dedicated J-Link debugger in order to reproduce the issue and to test any 'fixed' version of defmt RTT. See: https://www.segger.com/products/debug-probes/j-link/models/other-j-links/st-link-on-board/