MQTT subscribe does not work

[star297]

Has anyone actually got the mqtt subscribe function to work?

The publish works fine, however if I set

fona.MQTT_subscribe(SUB_TOPIC, 0); // Topic name, QoS

and do this (example from another post):

while (1) {

    char replybuffer[100];
    uint8_t idx = 0;

    if (fona.available()) {
      memset(replybuffer, 0, sizeof(replybuffer)); // Clear buffer
    }

    while (fona.available()) {
      replybuffer[idx] = fona.read();
      idx++;
    }

    if (idx > 0) {
      if (strstr(replybuffer, "+SMSUB:") != NULL) {
        Serial.print(F("MQTT message received: "));

        // Parse the message, splitting it at the quotes
        // +SMSUB: "topic","message"
        char *tok = strtok(replybuffer, "\""); // Skip the "+SMSUB: " part
        char *topic = strtok(NULL, ",\"");
        char *message = strtok(NULL, "\"");

        char msgBuffer[50]; // For holding the message
        strcpy(msgBuffer, message);

        // Do something based on what the message was
        Serial.println(msgBuffer); // Debug

      }
    }
  }

then publish data that is picked up on other device's, this just just sits there then after 60 seconds disconnects (as it should).

I tried to do it manually using AT commands, but still nothing, there's no data on the serial lines at all. Tried Qos 1 just in case. It seems to me that SIMCOM go out their way to hide information and make things difficult, or is just me?

Any ideas or suggestion's? I can go back to using the PUBSUB library if not.

[botletics]

The way the subscribe works is that there should be a "+SMSUB: " message that the SIM7000 sends out in serial when a message comes in for a topic you've subscribed to. You can test this with a raw USB-only connection using a serial terminal, or with the unmodified LTE_Demo sketch with the serial monitor open. If you see the "+SMSUB: " message appear, then you can proceed to test the Arduino code that parses it (maybe print it out first before doing any strtok() functions on it). I haven't personally tested that code yet.

[star297]

Based on Timothy's example above.

This works, I'm using a random subscribe topic and testing for "rls" to override the example's samplingRate. I have a 20 second timeout, if you send the topic 'retained' it will pick it up the next time it starts. But you will have to clear the retained flag on your MQTT broker. You could check for a carriage return to break the fona.available loop, but this works reliably. Needs that one millisecond, delay(1); on the ESP32 otherwise it will miss the data.

I can do the same if not more that the PUBSUB library as we can set Qos on the SIM7000. You will have to set up a function for servicing the subscribe messages automatically in a call back function, that may be a bit tricky. Perhaps Timothy could build it in to the library :)

  #define   SUB_TOPIC       "cc/dev/NB-IoT/rls"

  // subscribe to topic
  fona.MQTT_subscribe(SUB_TOPIC, 0); // Topic name, QoS

  char replybuffer[100];
  uint8_t idx = 0;
  int timeout = millis() + 20000; // 20 second timeout

  Serial.println("waiting replay");

  while (timeout > millis())
  {
    if (fona.available()) {
      memset(replybuffer, 0, sizeof(replybuffer)); // Clear buffer

      while (fona.available()) {
        replybuffer[idx] = fona.read();
        idx++;
        delay(1); // allow time for next character before ending loop
      }

      if (idx > 0) {
        if (strstr(replybuffer, "+SMSUB:") != NULL) {
          Serial.println(replybuffer);

          // Parse the message, splitting it at the quotes
          // +SMSUB: "topic","message"
          char *tok = strtok(replybuffer, "\""); // Skip the "+SMSUB: " part
          char *topic = strtok(NULL, "\"");
          char *message = strtok(NULL, ",\"");

          Serial.printf("MQTT message received:\n");
          Serial.println(topic);
          Serial.println(message);

          char msgBuffer[idx + 1]; // For holding the message
          strcpy(msgBuffer, message);
          idx = 0; // reset index counter

          if (strstr(topic, "rls") != NULL) { // look for something in the reply topic.

            samplingRate = atoi(msgBuffer);  // , setting the sampllingRate here

          }
        }
      }
    }
  }

[teddylambert]

@star297 With your code above you're able to read messages in from the SIMCOM? Having the same issue where no matter what I subscribe to, nothing comes through the SIMCOM (fona.available() is never returning true). I have that fona.available() and read functionality as the only code in my loop so it shouldn't be missing any messages...

I run the subscribe function in my setup and get an "OK" back so I think the AT+SMSUB message is working at least

Update: If I initiate the pub and sub from the same device, I'm able to get the +SMSUB message. But trying to publish a message from the test mosquitto broker and have my microcontroller pick it up (it's definitely the same topic) doesn't work. I'm wondering if it's something with SIMCOM--there's been a few times where I'll send AT+SMCONN and get "ERROR" back but if I still try to send a "publish" message after that error I can still see the message on the Mosquitto client which doesn't seem right

[star297]

The SIMCON MQTT does work, however its very slow compared using regular TCP data and PUBSUB. Its a Modem issue not the library. Also the Fona library needs UART interrupt and buffer handling to have an effective callback function otherwise you have to handle other functions like GPS when the messages come in or you will loose data. There's also a problem in the library MQTT subscribe function,

fona.MQTT_subscribe("cc/dev/13691592/cmdr", 0); // Topic name, QoS

will give a stack error crash unless you increase the container to 64 in line 2703, Adafruit_FONA.cpp

char cmdStr[32];

32 bytes is not enough.

I've been using this below with a 'mosquito' broker on a Raspberry pi Zero with MQTT exploer to monitor MQTT traffic and works fine.

void loop() {

  if (fona.available()) {
    while (fona.available()) {
      MQTTdata[idx] = fona.read();
      idx++;
      delay(1); // allow time for next character before ending loop
    }
    MQTTdata[idx] = '\0';

    if (idx > 0) {
      if (strstr(MQTTdata, "+SMSUB:") != NULL) {
        // Serial.println(MQTTdata);

        // Parse the message, splitting it at the quotes
        // +SMSUB: "topic","message"
        char *tok     = strtok(MQTTdata, "\""); // Skip the "+SMSUB: " part
        char *topic   = strtok(NULL, "\"");
        char *message = strtok(NULL, ",\"");

       // Serial.printf("MQTT message received:\n");
       // Serial.println(topic);
       // Serial.println(message);

        idx = 0; // reset index counter

        if (strcmp (topic, "cc/dev/sd") == 0) {   // ping request
          if (message[0] == 49) {   //  1
            Serial.printf("All devices Ping data request\n");
            sendDATA();
            Serial.printf("sent data back\n");
          }
        }

        if (strcmp (topic, "cc/dev/13691592/rls") == 0) {   // set lock request
          if (message[0] == 49) {   //  1
            lockCART();
          }
          if (message[0] == 48) {   //  0
            unlockCART();
          }
        }

        if (strcmp (topic, "cc/dev/13691592/cmdr") == 0) {   // commands

          if (message[0] == 50) {   //  2
            Serial.printf("Ping data request\n");
            sendDATA();
            strcpy(Path, MQTTdata_path);
            strcat(Path, "cmds");
            sprintf(MQTTdata, "%d" , 0);
            // Parameters for MQTT_publish: Topic, message (0-512 bytes), message length, QoS (0-2), retain (0-1)
            if (!fona.MQTT_publish(Path, MQTTdata, strlen(MQTTdata), 0, 0)) Serial.println(F("Failed to publish!"));
            Serial.printf("sent Ping ack back\n");
          }

          if (message[0] == 49) {   // ping sendDATA request
            Serial.printf("Ping data request\n");
            sendDATA();
            Serial.printf("sent data back\n");
          }
        }
      }
    }
  }
}

[teddylambert]

There's also a problem in the library MQTT subscribe function,

I changed up the formatting a bit so I don't need the cmdStr[32] so that a stack crash shouldn't occur.

I've been using this below with a 'mosquito' broker on a Raspberry pi Zero with MQTT exploer to monitor MQTT traffic and works fine.

Just to confirm, you're using the MQTT Explorer to subscribe and publish messages? Have my loop really simple right now

void loop() {
    /* Read published data */
    if(setupSuccess) //setup() code contains all of the parameters needed to enable MQTT
    {
        if (simcom.available())
            Serial.println("found message");
    }
}

and am seeing that the simcom.available() is just never triggering (simcom is just the same as your fona object). If I instead just Serial.print(simcom.read()) in my loop I get an endless string of 1-1-1-1-1-1....

Publish works beautifully so I know I'm able to establish communication with the broker, but messages back to the device (I'm using a MoteinoMega with the SIM7000 breakout board) when published from a different client don't seem to be coming through.

I'm running the SIMCOM at 115200 baud, but using hardware serial so to my knowledge the board should be able to handle that. Unless just in general 115200 is too fast for the MQTT connection? Update: Went ahead and tried switching the baud rate, still no success.

'm wondering if it's something with SIMCOM--there's been a few times where I'll send AT+SMCONN and get "ERROR" back but if I still try to send a "publish" message after that error I can still see the message on the Mosquitto client which doesn't seem right

Also ignore this earlier statement, wasn't realizing there were a few times I was retrying the program quick enough that the MQTT connection was staying active.

[codemasterJyri]

@star297, you were talking about using PUBSUB library with SIM7000 for the MQTT connection. Which library (or method) did you use to establish the TCP connection?

[star297]

I use the TinyGSM library: https://github.com/vshymanskyy/TinyGSM

This supports MQTT connection through pubsubclient: https://github.com/knolleary/pubsubclient

This does not have SSL connection whereas the SIMCOM built in MQTT does have this available.

TCP GPRS does work a little better, more responsive than the SIMCOM built in MQTT function. But TinyGSM does not directly support the SIMCOM built in MQTT functions.

I have got MQTT working here but needs a 'callback' function similar to what pusubclient has. If Tim could stitch this into the library it would be great for a GSM/GPRS only MQTT solution, no other library necessary.

[codemasterJyri]

Thank you, @star297 .I got the TinyGSM MqttClient.ino example working. But if I added my own code I started to see some really bizarre / weird behavior with this library. Added some functionality which is working just fine with Ethernet Shield + PubSubClient library, but somehow this code interrupts the normal behavior of the TinyGSM library. If the example code is able to normally connect to the mobile network (this code is located in the setup part of the code) then if I add some things into the loop the code is no longer even able to connect to the network (nonetheless connect to the server or publish / subscribe messages). I am just amazed. This seems utterly illogical to me as the Loop part comes after the Setup, meaning that the Setup part of the code should still work. How can this be happening? Also I am getting some: "[255298] ### Unhandled: " messages in the Serial Monitor. What do these mean and where do they come from? And this is weird that they "pop-up" at any given moment. Is there some kind of "threading" simulated in the background or do these messages come from the SIM7000 module? This is the first time in my coding experience that part of the code which otherwise works and code (which should not be reached yet) mess with each other. Could this be a low memory issue?

Sketch uses 27054 bytes (83%) of program storage space. Maximum is 32256 bytes.
Global variables use 1432 bytes (69%) of dynamic memory, leaving 616 bytes for local variables. Maximum is 2048 bytes.

Here's my output from the serial montitor:

Turning the modem on...
Module turned on!
Wait...
Initializing modem...
[24290] ### Unhandled: 
Modem Info: SIM7000E R1529
Waiting for network... fail
[106752] ### Unhandled: 
[107764] ### Unhandled: 
Network disconnected
[108773] ### Unhandled: 
[109785] ### Unhandled: 
[110797] ### Unhandled: 
[112056] ### Unhandled: 
[113068] ### Unhandled:
etc.

[star297]

You need to post this question on TinyGSM site really.

Afaik you can only have one instance of pubsubclient running at any one time. I think that's where your Unhandled errors are coming from. If using ethernet, you would need to disconnect the MQTT connection first before MQTT connecting using Wi-Fi or GSM. You can have more than one socket open so you don't need to disconnect the GSM GPRS data but only MQTT connection. Set your pubsubclient for each connection device, like this;

PubSubClient WiFiMQTT(espClient);
PubSubClient gsmMQTT(gsmClient);

In your code, you can do this before changing MQTT connection:

if (WiFiMQTT.connected()) {
      WiFiMQTT.disconnect();
      delay(100);
    }
// now process your GSM MQTT connection code.

if (gsmMQTT.connected()) {
      gsmMQTT.disconnect();
      delay(100);
    }
//  now process your WiFi MQTT connection code.

[codemasterJyri]

As I can understand - my explanation was not the best. I am not using the Ethernet and the GSM connection at the same time - I was just using the PubSubClient part + my own logic with the Ethernet Shield (at first, just to test it out) and after that I ported this work into the TinyGSM code.

But the good news is that I was able to solve the problem! I moved everything to Arduino Mega and the code is working! The code was not working with an Arduino UNO - therefore I assume that all the weird behavior was caused by low memory or the specific Arduino UNO board is somehow faulty. Actually just came up with a third explanation - I was using Software serial speed 9600 - this may also be the cause? I am not sure. All of these hypothesis are not proven yet. The only thing I know is that with an Arduino Mega (and using a hardware serial on that) everything is working flawlessly!