HELP begin multi path stream on large json

[denysmomo]

hi, i want access to my real time database for read some childpath. when call back for the first time stream print json like 5.5kb print correctly Serial.println("path: " + stream.dataPath + ", type: " + stream.type + ", value: " + stream.value);

if json have data like 7kb

[V][ssl_client.cpp:53] start_ssl_client(): Free internal heap before TLS 161672
[V][ssl_client.cpp:55] start_ssl_client(): Starting socket
[V][ssl_client.cpp:90] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:99] start_ssl_client(): Setting up the SSL/TLS structure...
[I][ssl_client.cpp:153] start_ssl_client(): WARNING: Use certificates for a more secure communication!
[V][ssl_client.cpp:177] start_ssl_client(): Setting hostname for TLS session...
[V][ssl_client.cpp:192] start_ssl_client(): Performing the SSL/TLS handshake...
[V][ssl_client.cpp:213] start_ssl_client(): Verifying peer X.509 certificate...
[V][ssl_client.cpp:222] start_ssl_client(): Certificate verified.
[V][ssl_client.cpp:237] start_ssl_client(): Free internal heap after TLS 120276
[V][ssl_client.cpp:276] send_ssl_data(): Writing HTTP request...

Global Stream Data streamCallback...
path: /, type: undefined, value: {"device":{"999evjBX.......          --> 7.2kb

and aftrer

74052
abort() was called at PC 0x401a86df on core 1

if json have data like 8-12kb

[V][ssl_client.cpp:53] start_ssl_client(): Free internal heap before TLS 161816
[V][ssl_client.cpp:55] start_ssl_client(): Starting socket
[V][ssl_client.cpp:90] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:99] start_ssl_client(): Setting up the SSL/TLS structure...
[I][ssl_client.cpp:153] start_ssl_client(): WARNING: Use certificates for a more secure communication!
[V][ssl_client.cpp:177] start_ssl_client(): Setting hostname for TLS session...
[V][ssl_client.cpp:192] start_ssl_client(): Performing the SSL/TLS handshake...
[V][ssl_client.cpp:213] start_ssl_client(): Verifying peer X.509 certificate...
[V][ssl_client.cpp:222] start_ssl_client(): Certificate verified.
[V][ssl_client.cpp:237] start_ssl_client(): Free internal heap after TLS 120428
[V][ssl_client.cpp:276] send_ssl_data(): Writing HTTP request...

Global Stream Data streamCallback...
abort() was called at PC 0x401a86df on core 1

if json have data like over 18kb the first call back not work on poweron

[V][ssl_client.cpp:53] start_ssl_client(): Free internal heap before TLS 161820
[V][ssl_client.cpp:55] start_ssl_client(): Starting socket
[V][ssl_client.cpp:90] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:99] start_ssl_client(): Setting up the SSL/TLS structure...
[I][ssl_client.cpp:153] start_ssl_client(): WARNING: Use certificates for a more secure communication!
[V][ssl_client.cpp:177] start_ssl_client(): Setting hostname for TLS session...
[V][ssl_client.cpp:192] start_ssl_client(): Performing the SSL/TLS handshake...
[V][ssl_client.cpp:213] start_ssl_client(): Verifying peer X.509 certificate...
[V][ssl_client.cpp:222] start_ssl_client(): Certificate verified.
[V][ssl_client.cpp:237] start_ssl_client(): Free internal heap after TLS 120564
[V][ssl_client.cpp:276] send_ssl_data(): Writing HTTP request...
[V][ssl_client.cpp:276] send_ssl_data(): Writing HTTP request...
[V][ssl_client.cpp:276] send_ssl_data(): Writing HTTP request...

Global Stream Data streamCallback...
path: /device/999evjBX99r2vHsIy3rewermdvAQVWhlkdBmtZSFZ0IeW/info, type: string, value: dfhbib
110012

not print json but work!
the problem is when i start new database and add some data in future i have a problem when esp32 poweron and my data have lengh from 6kb to 18kb :(

my code

FirebaseData globalFirebaseData;
String globalParentPath = "/global";
String globalChildPath[1] = {"/"};
size_t globalChildPathSize = sizeof(globalChildPath)/sizeof(globalChildPath[0]);

void globalStreamCallback(MultiPathStreamData stream);
void globalSstreamTimeoutCallback(bool timeout);

void setup() {
  if (!Firebase.beginMultiPathStream(globalFirebaseData, globalParentPath, globalChildPath, globalChildPathSize))
  Serial.println("error stream REASON: " + globalFirebaseData.errorReason());
  Firebase.setMultiPathStreamCallback(globalFirebaseData, globalStreamCallback, globalSstreamTimeoutCallback);
  while (true);
}

void globalStreamCallback(MultiPathStreamData stream)
{
  Serial.println();
  Serial.println("Global Stream Data streamCallback...");
  size_t numChild = sizeof(globalChildPath)/sizeof(globalChildPath[0]);
  for(size_t i = 0;i< numChild;i++)
  {
    if (stream.get(globalChildPath[i]))
    {
      Serial.println("path: " + stream.dataPath + ", type: " + stream.type + ", value: " + stream.value);
      Serial.println(ESP.getFreeHeap());
    }
  }
}
void  globalSstreamTimeoutCallback(bool timeout){if(timeout){}}

can someone help me? Thanks

PS. for now the solution is keep some casual data (minimum 20kb) on some child node

[mobizt]

The stream data (JSON in this case) will be stored in memory. As the size of JSON go bigger, your free heap will be reduced. If the free heap is running out, the problem will occur.

Handling of the large JSON stream data must be avoided because you are working with embedded device that its memory was limited.

You have only 161 k bytes before start the SSL connection which it may use approx. 100 k for SSL connection and only 60k bytes remains for use as stack and heap.

The reserved stack for the internal rtos task used in this library is 10000 bytes, which results in lower stack and heap available below 5000 bytes.

You should reduce the size of live data (stream) and keep it small as possible. It should be the sensor status or something that use only the number or boolean for storing.

Try to rename the node name to be shorter to reduce the size of JSON.

The long node name (key) like this "999evjBX99r2vHsIy3rewermdvAQVWhlkdBmtZSFZ0IeW" is useless and wasted your useful memory.