JS outgoing payload mapper doesn't know if payload is base64-encoded

Summary

Live message value in JS (argument value in mapFromDittoProtocolMsg function) outgoing payload mapper is sometimes base64-encoded, sometimes not. It seems to be dependent on content-type: if payload is TEXT or JSON, then payload is left intact, otherwise it's base64-encoded. There is no way for JS mapper to know it other than check content-type.

Details

My use case is following: I want to allow sending any headers in the message, for example, all headers starting with "x-" (e.g., "x-my-header") should be forwarded as message headers (MQTT 5 in my setup) without "x-" prefix (e.g., "my-header"). The payload should not be changed. Messages might be sent with any content-type. AFAIK, the easiest way to do this is by using JS payload mapper.

I noticed that depending on content-type, that the message is sent with, its payload might be base64-encoded. Looks like, the logic for this is:

    // messages/model/src/main/java/org/eclipse/ditto/messages/model/signals/commands/MessagePayloadSerializer.java
    static <T> void serialize(final Message<T> message, final JsonObjectBuilder messageBuilder,
            final Predicate<JsonField> predicate) {

        final Optional<ByteBuffer> rawPayloadOptional = message.getRawPayload();
        final Optional<T> payloadOptional = message.getPayload();
        final ContentType contentType = message.getContentType().map(ContentType::of).orElse(ContentType.of(""));
        final JsonValue payloadValue;
        if (rawPayloadOptional.isPresent() && !payloadOptional.filter(JsonValue.class::isInstance).isPresent()) {
            final ByteBuffer rawPayload = rawPayloadOptional.get();
            if (MessageDeserializer.shouldBeInterpretedAsTextOrJson(contentType)) {
                payloadValue =
                        interpretAsJsonValue(new String(rawPayload.array(), StandardCharsets.UTF_8), contentType);
            } else {
                final ByteBuffer base64Encoded = BASE64_ENCODER.encode(rawPayload);
                payloadValue = JsonFactory.newValue(new String(base64Encoded.array(), StandardCharsets.UTF_8));
            }
        } else if (payloadOptional.isPresent()) {
            final T payload = payloadOptional.get();
            payloadValue = payload instanceof JsonValue
                    ? (JsonValue) payload
                    : interpretAsJsonValue(payload.toString(), contentType);
        } else {
            payloadValue = null;
        }
        injectMessagePayload(messageBuilder, predicate, payloadValue, message.getHeaders());
    }

    // messages/model/src/main/java/org/eclipse/ditto/messages/model/signals/commands/MessageDeserializer.java
    public static boolean shouldBeInterpretedAsTextOrJson(final ContentType contentTypeHeader) {
        return contentTypeHeader.isText() || contentTypeHeader.isJson();
    }

But in payload mapper there is simply no way to find out if the payload was encoded. No way other than checking content-type by itself. What I did to work this around is:

created my own JS function (inside payload mapper code) that determines if content-type is text or json according to logic in java class ContentType
used it to return either textPayload from value as is, or bytePayload as dcodeIO.ByteBuffer.fromBase64(value).toArrayBuffer()

Expected Result

I'd like this behavior to be documented, for example under Mapping outgoing messages section on Payload Mapping page.
It would be great to have function, e.g., Ditto.isTextContentType/Ditto.isJsonContentType, or Ditto.isPayloadBase64Encoded, that will reflect code from java class ContentType that decides parsing strategy based on content-type.

eclipse-ditto / ditto