stream seen on logging but doesn't deserialize, no errors

[paulmartinetti]

Salut, first I posted my question on stack, completely stumped. Merci d'avance. I see the stream coming in correctly but I can't fetch the values from doc["key"] as it's not parsing and I get no error. https://stackoverflow.com/questions/66283877/esp32-wificlient-arduino-stream-json-no-errors-cant-fetch-values

[bblanchon]

Hi @paulmartinetti,

You already had some responses on StackOverflow. Did you try them?

Would you mind rephrasing your question here? Please also state the current status of your research and provide a short code sample (max 50 lines).

Best regards, Benoit

[paulmartinetti]

I'm connecting an esp32 WiFiClient to a simple socket server in nodejs, where we configure the project. The client prints the stringified json in the Serial monitor using your streamUtils or even just char = client.read(); which digests 1 byte at a time. But when I fetch the deserialized values from doc, doc = 1 or 0, the key:value pairs are gone and no errors. If I ask for char[15] it returns the correct byte. If I deserialize the char, no errors, can't fetch values. I've tried slowing it down with delays, I tried deserializeMsgPack, tried both static and dynamic json doc, I used the Assistant to calculate memory to reserve and even tripled that value. We can try other transfer protocols, but I've never received the data like this, seen it, and not been able to use it. Thanks for your time.

#include <WiFi.h>
#define ARDUINOJSON_ENABLE_ARDUINO_STREAM 1
#include <ArduinoJson.h>
#include <StreamUtils.h>

void setup () {
//connnect to wifi
}
...
StaticJsonDocument<384> doc;
DeserializationError error;
...
void loop () {
// Use WiFiClient class to create TCP connections
  WiFiClient client;
  const int httpPort = 1337;
  if (!client.connect(host, httpPort)) {
    Serial.println("connection failed");
    return;
  }
client.print("");
while (client.available() > 0) {
    ReadLoggingStream loggingClient(client, Serial);
    error = deserializeJson(doc, loggingClient);
  }
if (error) {
    Serial.print(F("deserializeJson() failed: "));
    Serial.println(error.f_str());
    return;
  }
int id = doc["id"]; // 5
Serial.print("id: "); Serial.println(id);
} .. end loop

Serial monitor

07:16:36.574 -> connecting to 10.0.0.250
07:16:36.849 -> "{\"id\":5,\"nom\":\"whynot\",\"delayStart\":200,\"rampePWM\":11,\"pulseWelding\":200,\"speedBalayage\":0.4,\"speedWelding\":0.5,\"speedWire\":1.1,\"balayage\":0.8,\"pulseWire\":5,\"retractWire\":7}"
07:16:36.849 -> id: 0

[bblanchon]

Hi @paulmartinetti,

The problem is that the node backend stringifies the JavaScript object twice. In other words, it does something like this:

JSON.stringify(JSON.stringify({id:5,nom:'whynot',delayStart:200}))

So, instead of returning a JSON object, it returns a JSON string value that contains a JSON document.

Expected:

{"id":5,"nom":"whynot","delayStart":200}

Actual:

"{\"id\":5,\"nom\":\"whynot\",\"delayStart\":200}"

Since you didn't share the JavaScript code, I can only assume that you explicitly called JSON.stringify(), but the framework already converts to JSON implicitly (for example, res.json(obj) with Express). If that's the case, simply remove the call to JSON.stringify(), and you should be good to go.

If you cannot alter the backend, you can parse the inner JSON document by calling deserializeJson() twice, like so:

StaticJsonDocument<1024> doc1, doc2;

deserializeJson(doc1, client);
deserializeJson(doc2, doc1.as<const char*>());

Best regards, Benoit

[bblanchon]

PS: I added the following page to the documentation.

• deserializeJson() succeeds but I cannot read any value

[paulmartinetti]

It works! You are correct, I changed the server method to socket.write(data); and it works. Thank you!

[paulmartinetti]

The working node express socket is:

var net = require('net');
var serverN = net.createServer(function(socket) {
    fs.readFile("./data/projet.json", 'utf-8', (err, data) => {
        if (err) {
            throw err;
        }
        socket.write(data); // <-- this used to be socket.write(JSON.stringify(data));
        socket.pipe(socket);
    });
});

serverN.listen(1337, '10.0.0.250');

[bblanchon]

You're welcome, @paulmartinetti. Thank you for using ArduinoJson. Good luck with your project!