How to transmit binary data?

[BenediktBurger]

Sometime you have to transmit binary data, like an image. JSON (our main encoding scheme) does not support binary data. What do we recommend?

Several options, which are all compatible with current LECO definitions:

1. You can send the binary data in the first (and more) payload frame, while setting the message_type to some specific (TBD) value
2. You can send a JSON message in the first payload frame and add the binary objects in additional payload frames. This requires, however, that the code looks at the whole message and makes the additional frames available to the json-rpc interpreter somehow. Here it might be good to define its own message type as well. (I implemented this for PyMoDAQ)
3. You can encode the binary data with some binary-to-text encoding to plain ASCII text, which can be sent via JSON. The most common encoding seems to be base64.

All three approaches have their advantages and disadvantages. I guess the choice depends on the circumstances. Here I want to present them, maybe we can recommend one or the other, or even all three as

[BenediktBurger]

More elaborate discussion of the different methods.

1. Binary data as the only payload

Implementation

The payload of the LECO message is the binary data, without any (pre-defined) other information.

Pro

From the LECO side quite simple: put the binary data in one or several payload frames (several for several binary objects).

Cons

• It requires an additional (or even several different) message types, to indicate how to interpret the data. Maybe even user defined message types.
• Methods cannot be described via Open-RPC descriptions of JSON-RPC methods.
• It works only as a response to a request, as the requester knows, what he requested and can attribute the response to the original request via the conversation id
• Difficult to use the binary data as a parameter of a RPC call, as it does not contain the information which method to call.
• From the experience and point of view of PyLECO: it requires additional logic, as the default implementation uses JSON-RPC natively.

2. JSON message and additional payload

Implementation

• The first payload frame contains normal JSON-request/response. The binary data is in additional payload frames, which are unused for JSON-RPC.
• A method is defined without the binary data as parameter and/or return value as it does not receive the binary data via JSON-RPC nor does it return it via JSON-RPC. Instead, the method has to access the binary data somewhere else and indicate the binary return value somewhere else, as well.

example mehod (based on PyLECO):

current_message: Message  # contains the LECO message
additional_response: bytes  # will later contain the additional, binary, response

def method_with_binary(json_parameter):
    binary_data = current_message.payload[1]
    result = do_something(json_parameter, binary_data)
    additional_response = result
    return None

# afterwards the additional_response has to be added to the response message before sending

Pros

• Content of first payload frame is defined.
• JSON message can be interpreted by present JSON-RPC infrastructure
• You can give normal JSON-RPC compliant parameters to a method additionally to the binary data

Cons

• Methods cannot be described completely by Open-RPC description of JSON-RPC methods as the binary data parameter/return values are not mentioned (only in the docstring)
• More obscure methods, as the binary data does not enter the signature nor return value.

Open questions

• How does a method know, whether it should look for binary data in additional frames?
  • With a special message type?
  • Because it always does?
  • Because a parameter has a certain value and there are additional frames?
• How does the recipient of the response know, that the JSON response is not the real response?
  • With a special message type?
  • Because the response has a certain value (which one? None?) and there are additional frames?
  • Because it knows, that it requested binary data? but the JSON-RPC-decoding method (in PyLECO) does not know it

3. Encode binary data to string and send it via JSON

Implementation

Encode your binary data to a string and use that as a parameter for JSON-RPC.

Pros

• Uses plain JSON-RPC (and Open-RPC definitions)
• No additional logic in the LECO elements needed, just encoding/decoding in the sender/receiver.
• Readily available algorithms and their libraries (e.g. base64) for encoding.

Cons

• Method definitions contain "string" instead of the binary data type as parameter name
• Encoding binary data to a (ASCII) string increases data size (by a third for base64) and takes computation time. That is especially noteworthy, as binary data tends to be comparatively large (oscilloscope trace, camera image...).

Comments

• Using only the binary data (option 1) is the most simple in principle, but requires the most additional effort (manual message handling)
• Using additional payload frames (option 2) adds some complexity, albeit limited, but offers binary data natively.
• Encoding binary data as a string (option 3) is the most simple one to implement without changing the message handling system. This is implemented in PyMoDAQ.
• For simple setups with small binary data, I recommend option 3 due to the reduced complexity (just encoding/decoding added).
• For optimized data transfer, I recommend option 2.