Encoding to string and decoding changes the data.

[BlackEdder]

As part of our mesh library we use ArduinoJson for all messaging. This works very well for most of the messages, but there is one message type where json is just to inefficient (space wise), so I am trying to optimise that by serialising and deserialising the data, before adding it to the json message. This works fine, until I convert the resulting JsonObject to a string and then decode it again. Doing that seems to change the data. See below for a minimal example (the problematic case is at the bottom). Any help/pointers would be appreciated.

#include <ArduinoJson.h>

#define htonl(x) __builtin_bswap32((uint32_t) (x))
#define ntohl(x) __builtin_bswap32((uint32_t) (x))

void serialize(char *outBuf, uint32_t data) {
    auto rdata = htonl(data);
    memcpy(outBuf, &rdata, sizeof(data));
}

uint32_t deserialize(const char *data) {
    uint32_t result;
    memcpy(&result, data, sizeof(result));
    uint32_t rresult = ntohl(result);
    return rresult;
}

void setup() {
    Serial.begin(115200);
}

void loop() {
    delay(1000);
    Serial.printf("Test\n");
    uint32_t a = 128299;
    char outBuf[sizeof(a) + 1];

    serialize(outBuf, a);
    outBuf[4] = '\0'; // null terminate the string.
    Serial.printf("a: %u\n", a);
    Serial.printf("a_str: %s\n", outBuf);

    uint32_t b;
    b = deserialize(outBuf);
    Serial.printf("b: %u\n", b);

    // JSON
    DynamicJsonBuffer jsonBuffer;

    // This still works
    JsonObject& root = jsonBuffer.createObject();
    root["subs"] = outBuf;
    const char *outBuf2 = root["subs"];
    b = deserialize(outBuf2);
    Serial.printf("b2: %u\n", b);

    // WILL NOT WORK
    // Encoding the object to a json string and then decoding it
    // does not work
    String ret;
    root.printTo(ret);
    JsonObject& root2 = jsonBuffer.parseObject(ret);
    root2.printTo(Serial);
    const char *outBuf3 = root2["subs"];
    b = deserialize(outBuf3);
    Serial.printf("b3: %u\n", b); // This should print out 128299, but it doesn't
}

Some further details:

• This is on a nodemcuv2
• I use platformio to manage the project.

[BlackEdder]

I have worked out that this (probably) is because outBuf starts with a '\0'. Is there anyway to make arduinojson encode the whole character array (including '\0'), for example by giving it the length of the buffer to copy?

[bblanchon]

Hello Edwin,

I'm not sure I understand what you're trying to accomplish. Are you trying to put binary data (possibly containing zeros) into a JSON string? If so, why don't you use base-64 encoding?

Regards

[BlackEdder]

If so, why don't you use base-64 encoding?

Short answer, because I didn't know that was a standard way of doing it. Longer answer, because base64 encoding takes at least 1.5 times as many characters as the base256 encoding I was trying to accomplish :)

[bblanchon]

Base-64 is the usual way to store binary data in JSON.

I'm afraid there is nothing I can do. Can we close this issue?

[BlackEdder]

Yes, thank you for the feedback :)