[RFC] Data multiplex & protocol

[dzarda]

There are three logical data streams:

• CDC to coprocessor
• CDC to ESP tunnel
• ESP to coprocessor

One option is to open two CDC channels, but I think it does more bad than good and only partially solved the problem.

For simplicity sake we probably want to share the protocol on both CDC and downstream UART. Protobuf can be used to describe the whole data domain.

message CoprocIn {
  ...
}

message CoprocOut {
  ...
}

message Passthrough {
  bytes content = 0;
}

message Packet {
  oneof type_oneof {
    CoprocIn = 0;
    CoprocOut = 1;
    Passthrough = 2;
  }
}

Does this cross OSI layers? Yes. Does it worry me? Not much because this is a very closed system without networking.

@cednik Is this similar to the bangbang-notransaction-architecture you described?

[yaqwsx]

I like CDC for commands and the second one for ESP32 tunnel. Why should it do more harm than good? It seems like a perfect solution to me - you don't need any special application to encapsulate tunnel communication on the computer side and it works out-of-the-box with all the existing software using a serial line.

When you have two CDCs, I would use only CoprocIn and CoprocOut without Packet - since you don't need it. There's no need to have a common type of messages for both directions of the communication.

Also, I am not sure if I understand the note about OSI layers. We're not building a network device so I see no reason to make it somewhat compatible with OSI.

[dzarda]

Good points. The OSI thing was OT really sry.

The two CDCs only solve the upstream problem. We have only one physical UART between coproc & ESP. Here we still have to mux the data somehow.

But the pure serial upstream tunnel is a killer feature and I agree with you, we should do it. Could bring just a slight confusion on the PC part - should I connect to COM4 or COM 6?

[Tasssadar]

I'm not sure I understand the way it is supposed to work, but I think I missed one meeting, so I apologize if following is just wrong.

Which UART on ESP32 is the coprocesor using, is it the "main" one that is connected to ESP32's USB? Or one of the other ones?

In any case, I don't see the point in tunneling ESP32<->PC. Since the UART is taken up by the coprocesor communication, the app on ESP32 can't use it (unless we somehow implement another "tunneled UART" in our own library, and then the app has to explicitly use that one).

The only reason I see for PC<->ESP32 tunnel is for programming the ESP. In that case, the coprocesor has to recognize that is what is happening and stop communicating via protobufers (also, if we are using one of the non-main UARTs, can ESP32 be programmed via that one?).

Another option is sending the new program to ESP32 via our protobuf protocol and letting it program itself (like OTA), but then you don't need the tunnel at all.

I think that you need two UARTs between ESP32 and coprocesor for this to work nicely.

As for the two COM ports, you are able to pass some device/manufacturer strings to the PC: https://github.com/RoboticsBrno/RB3204-RBCX/blob/fw-cdc/fw/rbcx-coprocessor/src/usb_cdc_link.c#L148-L150

If those are per-interface and not per device, they could be used to described each interface. I think that's how we recognize Shupito in Lorris.

BTW, protobufers optimized for MCUs: https://github.com/nanopb/nanopb

[yaqwsx]

The two CDCs only solve the upstream problem. We have only one physical UART between coproc & ESP. Here we still have to mux the data somehow.

Ok, I missed that. I thought we have 2 UARTS available.

If we only want to program ESP32 via the bridge, it is easy. Once you detect reboot sequence on the DTR/RTS pins (I hope I got them right), you reset ESP32 and switch to the pure tunnel. Once DTR/RTS returns to normal, you go back to a normal mode.

If we want also redirect the user UART (printf, std::cout & related), it could be done via funopen - basically you change the read & write functions for standard input/output. I see several possible drawbacks/open issues here:

• if you connect directly to ESP32 with USB cable you see a binary mess on the terminal
• you cannot transmit boot messages & backtraces in this way since they access UART directly via a function hard-coded in binary blob in drivers supplied by Espressif.
• I am not sure if Arduino uses standard input/output or talks to UART directly

The other option is to just frame our Probuf messages so that chance of confusing them with a regular text stream is negligible. Then the expander consumes all the /rotobufs and unrecognized parts of stream forwards to CDC.

The third option is to use an extra pin between the expander and ESP (not that I hope we have a spare pin @cednik :-P ) and use that to change UART meaning. But I see there a problem when crash happens during sending a protobuf message - we loose the backtrace.

[dzarda]

I stand corrected, theres:

• a dedicated ESP_PROG_/UART_ from coproc to ESP32 TXD0 & RXD0 presumably for programming (?)
• a shared EXPANDER-UART-/ from coproc to ESP32IO2&IO12`, also connected to some pinheaders.

Dunno why I thought there's just one pair.

[dzarda]

Closing - one UART for verbatim tunneling and second serves as coproc iface