How many items maximum can tb.sendTelemetry handle?

[leejames92]

Hi,

Prior to void setup(); I've set to ThingsBoardSized<128> tb(espClient);

I'm using ThingsBoard library version 0.6.0.

However the max items that I can send is only 6 as per below:

    const int data_items = 6;

    Telemetry data[data_items] = {
      { "Target", target_m },
      { "Distance", distance_m },
      {"Plantime", plan_mins},
      {"NSpeed", machspeed},
      {"ASpeed", current_mtrpermins},
      {"Downtime", downtime_mins},
    };

    tb.sendTelemetry(data, data_items);

If I want to successfully send 15 items, I'll have to split into 3 sections as the code below:

const int data_items2 = 5;
    const int data_items3 = 4;
    const int data_items = 6;

    Telemetry data[data_items] = {
      { "Target", target_m },
      { "Distance", distance_m },
      {"Plantime", plan_mins},
      {"NSpeed", machspeed},
      {"ASpeed", current_mtrpermins},
      {"Downtime", downtime_mins},
    };

    tb.sendTelemetry(data, data_items);

    Telemetry data2[data_items2] = {
      { "Down1", Dtime1 },
      { "Down2", Dtime2 },
      {"Down3", Dtime3},
      {"Down4", Dtime4},
      {"Down5", Dtime5},
    };

    tb.sendTelemetry(data2, data_items2);

    Telemetry data3[data_items3] = {
      { "Down6", Dtime6 },
      { "Down7", Dtime7 },
      {"Down8", Dtime8},
      {"Down9", Dtime9},
    };

    tb.sendTelemetry(data3, data_items3);

The above code works to send all 15 but when I want to push to Telegram, it won't stitch as one message but separated in to 3 messages.

I would like to know if I can have 15 data send one go from my Arduino Mega with ESP8266 hooked up.

I've also tried ThingsBoardSized<128,32> tb(espClient); but no difference seen.

Do hope there is a workaround for this.

Best regards,

James

[leejames92]

Also to add-on, if I use latest Thingsboard version 0.7.1, I have this error:

C:\Users\James\Documents\Arduino\libraries\ThingsBoard\src/ThingsBoard.h:17:10: fatal error: vector: No such file or directory Not used: C:\Program Files (x86)\Arduino\libraries\Adafruit_BusIO

include <vector>

[leejames92]

Hi,

Anyone? Sorry I really need clarification about this.

[leejames92]

Hi anyone?

[abferguson]

leejames92- Your issue may not be with the ThingsBoard Arduino SDK library but with the PubSub library. (I'm using the PubSubClient library from Nick O'Leary with Arduino IDE.

Try changing the value of MQTT_MAX_PACKET_SIZE (~line 26 of Nick O'Leary's) from 256 to something larger. This worked for me.

I found that it's the total number of characters in the data packet and not the number of ThingsBoard fields that causes this issue. Surprisingly, no error is thrown when the size is too large.

[MathewHDYT]

The old version of the library did not pass the size you set as a template argument ThingsBoardSized<128,32> to the PubSubClient constructor, this has been fixed in later version.

Additionally the first parameter means the amount of characters ('A', 'B', etc.) you need to send to ThingsBoard. I would recommend you to use the ArduinoJson #define called JSON_OBJECT_SIZE(15), if you pass the amount of data points you want to send into that. Meaning 15 it will automatically calculate how much space you need.

The second parameter is the allowed amount of keys you can send.

You can simply then pass those as template arguments instead ThingsBoardSized<JSON_OBJECT_SIZE(15), 15>.

Be aware this isn't always perfect, especially if you use long strings or a value is actually an array. For your use case this should work perfectly tough.

[abferguson]

Thanks for the tip on the ThingsBoardSized<> constructor!  I've made the change. I was following the example on the thingsboard-arduino-sdk site. I retrieved and worked with the second fork commit.   Both ESP8266 and ESP32 compiled.   The ESP8266 crashes while in setup().   The ESP32 runs fine with my minimal functionality example (test RPC and subscribed shared attribute functionality along with sendAtttributes functionality)

Order of things in my test sketch in setup():1-Connect to WiFi2-Subscribe to ThingsBoard (fail)Note: using PubSubClient 2.8.0 by O'Leary

For what it's worth, here's the Arduino IDE ESP Exception Decoder output for the ESP8266 crash- Exception 3: LoadStoreError: Processor internal physical address or data error during load or store PC: 0x40213d26 EXCVADDR: 0x40242f61

Decoding stack results 0x402059a0: PubSubClient::connect(char const, char const, char const, char const, unsigned char, bool, char const, bool) at C:\Users\ferguson\Documents\Arduino\libraries\PubSubClient\src\PubSubClient.cpp line 196 0x40100900: umm_init_common(size_t, void, size_t, bool) at C:\Users\ferguson\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\cores\esp8266\umm_malloc\umm_malloc.cpp line 428 0x40205b88: PubSubClient::connect(char const, char const, char const*) at C:\Users\ferguson\Documents\Arduino\libraries\PubSubClient\src\PubSubClient.cpp line 170 0x4020b8b5: PubSubClient::connected() at C:\Users\ferguson\Documents\Arduino\libraries\PubSubClient\src\PubSubClient.cpp line 690 0x40201dc0: subscribeThingsboard() at C:\Users\ferguson\Documents\Arduino\libraries\ThingsBoard\src/ThingsBoard.h line 426 0x40201e95: setup() at G:\workdir\Microcontrollers\Examples\ESP32\ThingsBoard\RPC_and_attribute_update_callbacks_thingsboard_0.7.1/RPC_and_attribute_update_callbacks_thingsboard_0.7.1.ino line 59 0x402086f0: loop_wrapper() at C:\Users\ferguson\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\cores\esp8266\core_esp8266_main.cpp line 198

On Wednesday, December 7, 2022 at 12:54:51 PM CST, MathewHDYT ***@***.***> wrote:  

The old version of the library did not pass the size you set as a template argument ThingsBoardSized<128,32> to the PubSubClient constructor, this has been fixed in later version.

Additionally the first parameter means the amount of characters ('A', 'B', etc.) you need to send to ThingsBoard. I would recommend you to use the ArduinoJson #define called JSON_OBJECT_SIZE(15), 15, if you pass the amount of data points you want to send into that. Meaning 15 it will automatically calculate how much space you need.

The second parameter is, that would be the allowed amount of keys you can send.

You can simply then pass that into the Constructor instead ThingsBoardSized<JSON_OBJECT_SIZE(15), 15>.

Be aware this isn't always perfect, especially if you use long strings or a value is actually an array. For your use case this should work perfectly tough.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[MathewHDYT]

Interesting according to the espressif documentation a LoadStoreError seems to happen if the code either attempts to store in IROM or DROM, or alternatively if a load/store is misaligned 8bit instead of 32 (dereferencing a pointer to an instruction).

Would it be possible for you to comment the setup method that crashes?

It seems to be the PubSubClient connect method that causes issues somehow.

[abferguson]

Here's my test code. I apologize but don't fully understand your question about commenting the setup(0 method that crashes. '''c++ / This sketch tests simultaneous     RPC callbacks     Shared attribute callbacks     Attribute sending    Using ThingsBoard SDK 0.7.1    Hardware: ESP32 & ESP8266/

include "ThingsBoard.h" /// Must make some edits to fix code and prevent duplicate const#include  /// ESP8266 only//#include  /// ESP32 only

/// Set the following four constants to your setupconst char ssid[] = "xxxxx";                     /// SSID of WiFiconst char wifiPassword[] = "yyyyy";        /// Password of WiFiconst char deviceToken[] = "zzzzz"; /// Access token of test deviceconst char thingsboardServer[] = "192.168.1.107";  /// IP address of ThingsBoard server//////////////////////// bool subscribed = false;bool attrsubscribed = false; bool toggleSwitchStatus = false; WiFiClient espClient;ThingsBoardSized<JSON_OBJECT_SIZE(10),10> tb(espClient); ///--------------------------------------------------void setup() {  Serial.begin(115000);  delay(1000);   initWiFi();  subscribeThingsboard();    Serial.println(F("End of setup()"));} /// void setup()///-------------------------------------------------- /// Set value on switchRPC_Response toggleSwitchSet(const RPC_Data &data){  Serial.println(F("in toggleSwitchSet"));  toggleSwitchStatus = !toggleSwitchStatus;    Attribute attributetogswset[1] = {    { "toggleswitchstatus", toggleSwitchStatus },  };  tb.sendAttributes(attributetogswset, 1);   return RPC_Response(NULL, toggleSwitchStatus);}; /// Get value on switchRPC_Response toggleSwitchGet(const RPC_Data &data){  Serial.println(F("in toggleSwitchGet"));  return RPC_Response(NULL, toggleSwitchStatus);} const std::vector callbacks = {  { "toggleswitchset", toggleSwitchSet },  { "toggleswitchget", toggleSwitchGet }}; ////// Shared attributesvoid processSharedAttributeUpdate(const Shared_Attribute_Data &data) {/// Next four lines based on Thingsboard example C++98 syntax/// Changed JsonObject to JsonObjectConst to work  for (JsonObjectConst::iterator it = data.begin(); it != data.end(); ++it) {    Serial.println(it->key().c_str());    Serial.println(it->value().as<char>()); // We have to parse data by it's type in the current example we will show here only char data  }   int jsonSize = JSON_STRING_SIZE(measureJson(data));  char buffer[jsonSize];  serializeJson(data, buffer, jsonSize);  Serial.print(F("Shared attribute buffer: ")); Serial.println(buffer);} /// void processSharedAttributeUpdate() const Shared_Attribute_Callback callback(processSharedAttributeUpdate);/// ///==================================================void loop() {  if (WiFi.status() != WL_CONNECTED)    reconnect();   subscribeThingsboard();  subscribeRPC();  subscribeSharedAttributes();   tb.loop();} /// void setup()///================================================== void initWiFi() {  Serial.println(F("initWiFi()"));  WiFi.mode(WIFI_STA);  WiFi.disconnect(true);  WiFi.begin(ssid, wifiPassword);  while ((WiFi.status() != WL_CONNECTED)) {    delay(20);  } /// while   Serial.println(F("End initWiFi()\n"));} /// void initWiFi() void reconnect() {  Serial.println(F("reconnect()"));  WiFi.disconnect(true);  WiFi.begin(ssid, wifiPassword);   while (WiFi.status() != WL_CONNECTED) {    delay(20);  }  Serial.println(F("End reconnect()"));} /// void reconnect() void subscribeThingsboard() {  Serial.println(F("subscribeThingsboard(): "));  if (!tb.connected()) {    subscribed = false;    attrsubscribed = false;     while (!tb.connect(thingsboardServer, deviceToken)) {      delay(10);    }    Serial.println(F("Subscribed to Thingsboard"));  }} /// void subscribeThingsboard() void subscribeRPC() {  if (!subscribed) {    Serial.println(F("subscribeRPC()"));    if (!tb.RPC_Subscribe(callbacks.cbegin(), callbacks.cend())) {      Serial.println(F(" Failed to subscribe to RPC"));      //delay(5000); /// Add delay to prevent rapid subscribing      return;    }    subscribed = true;    Serial.print(F("subscribed: ")); Serial.println(subscribed);  }} /// void subscribeRPC ///void subscribeSharedAttributes() {  if(!attrsubscribed) {    if (!tb.Shared_Attributes_Subscribe(callback)) {      Serial.println("Failed to subscribe for shared attribute updates");      delay(5000); /// Add delay to prevent rapid subscribing      return;    }     attrsubscribed = true;    Serial.println(F("End subscribeAttributes()"));  }} /// void subscribeAttributes()//*/ '''

On Wednesday, December 7, 2022 at 03:34:35 PM CST, MathewHDYT ***@***.***> wrote:  

Interesting a LoadStoreError seems to happen if the code either attempts to store in IROM or DROM, or alternatively if a load/store is misaligned 8bit instead of 32 (Dereferencing a pointer)

Would it be possible for you to comment the setup method that crashes?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[MathewHDYT]

Sorry for the confusion with commenting I just meant that you could post the code in this GitHub thread, like you did.

But for the future, if you paste code into GitHub. I would recommend you to use code button atop of the writing box, this allows to mark segments as code.

Just use three ` following the coding language used (c++) then a newline with all your code and at the end add a newline and three ` again to close that section.

So for example: ```c++ ThingsBoardSized<JSON_OBJECT_SIZE(10),10> tb(espClient); ```

---

For the code I'm confused about your thingsboardServer variable with the value 192.168.1.107, because that IP would be a private IP address, do you use the same thingsboardServer value for the ESP32 as well and it works there?

const char deviceToken[] = "zzzzz"; 
/// Access token of test device
const char thingsboardServer[] = "192.168.1.107";

void setup() {
    Serial.begin(115000);
    delay(1000);
    subscribeThingsboard();
    Serial.println(F("End of setup()"));
}

void subscribeThingsboard() {
  Serial.println(F("subscribeThingsboard(): "));
  if (!tb.connected()) {
    while (!tb.connect(thingsboardServer, deviceToken)) {
        delay(10);
    }
    Serial.println(F("Subscribed to Thingsboard"));
  }
}

---

Because according to the LoadStoreError you've pasted previously the PubSubClient seems to crash in this specific connect method:

boolean PubSubClient::connect(const char *id, const char *user, const char *pass) {
    return connect(id,user,pass,0,0,0,0,1);
}

and then even more specifically probably on the result = _client->connect(this->domain, this->port); line.

boolean PubSubClient::connect(const char *id, const char *user, const char *pass,
const char* willTopic, uint8_t willQos, boolean willRetain, const char* willMessage, boolean cleanSession) {
    if (!connected()) {
        int result = 0;

        if(_client->connected()) {
            result = 1;
        } else {
            if (domain != NULL) {
                result = _client->connect(this->domain, this->port);
            } else {
                result = _client->connect(this->ip, this->port);
            }
        }

---

The thing I'm assuming currently is that the PubSubClient can accept an IP but it can't be passed like it is done in your example. Might it be possible for you to change this line of the script and I will create a overload in a new commit, that allows to connect with a direct IP address as well.

const char thingsboardServer[] = "192.168.1.107";
// To this
IPAddress thingsboardServer = IPAddress(192, 168, 1, 107);

[abferguson]

My apologies for not using proper commenting. I very much appreciate your patience.

To be clear, the only combination of ESP8266/ESP32 and ThingsBoard.h versions that runtime fail in the Thingsboard subscribing function, using the line below, is ESP8266 + ThingsBoard.h 0.7.1. The ESP32 works.

const char thingsboardServer[] = "192.168.1.107";

---

Following your suggestion, I tried using the following two versions to define 'thingsboardServer'

IPAddress thingsboardServer = IPAddress(192,168,1,107); 
// and
IPAddress thingsboardServer(192,168,1,107);  // pulled from mqtt_basic example for PubSub.

with the resulting compile error for both versions below

error: conversion from 'IPAddress' to 'const char*' is ambiguous
  154 |     while (!tb.connect(thingsboardServer, deviceToken))

I note that in ThingsBoard.h, the way the connection using PubSub is done is different between v0.6 and 0.7. Could this be the issue? I'm running out of time today but might try to swap v0.6 into 0.7.

    inline const bool connect(const char *host, const char *access_token = PROV_ACCESS_TOKEN, int port = 1883, const char *client_id = DEFAULT_CLIENT_ID, const char *password = NULL) {

[MathewHDYT]

Sorry the change didn't work previously, because the code does not have an overload for IP Address class. I've added that with my newest commit to the fork tough, would be nice if you could try it and see if it compiles and especially if the runtime error is gone now or if it still persists.

[MathewHDYT]

@imbeacon I think this issue can be closed, previously the ESP8266 had errors with PROGMEM, therefore it is optional now and should work.