search

vshymanskyy / TinyGSM

A small Arduino library for GSM modules, that just works
GNU Lesser General Public License v3.0
1.91k stars 713 forks source link

SIM800l - MQTTS connection return status -2 #425

Open llaneiro opened 3 years ago

llaneiro commented 3 years ago

[X] I have read the Troubleshooting section of the ReadMe

[ ] Request to support a new module [ ] Bug or problem compiling the library [X] Bug or issue with library functionality (ie, sending data over TCP/IP) [X] Question or request for help

Modem: SIMCOM_SIM800L R14.18 Main processor board: ESP8266 TinyGSM Version: 0.10.5

#define TINY_GSM_MODEM_SIM800
#include <TinyGsmClient.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>
#include "keys.h"

SoftwareSerial SerialGSM(13, 15);  // RX, TX
TinyGsm modemGSM(SerialGSM);
TinyGsmClientSecure gsmClient(modemGSM);

PubSubClient MQTT(gsmClient);

//MQTT
void initMQTT() {
  MQTT.setServer(BROKER_MQTT, BROKER_PORT);
  MQTT.setCallback(callback);
}

void callback(char* topic, byte* payload, unsigned int length) {
  String msg;
  for(int i = 0; i < length; i++) {
    char c = (char)payload[i];
    msg += c;
  }

  Serial.print("Recived ");
  Serial.println(msg);
}

void reconnectMQTT() {
  while (!MQTT.connected()) {
    Serial.print("Trying to reconnect to the MQTT broker");
    Serial.print("Status: ");
    Serial.println(MQTT.state());
    initMQTT();

    if (MQTT.connect(ID_MQTT, MQTT_USERNAME, MQTT_KEY)) {
      Serial.println("Connected to the MQTT broker");
      // MQTT.subscribe(TOPIC_SUBSCRIBE);
    }
    else {
      Serial.println("Failed to reconnect to the broker");
    }
  }
}

void checkConnections() {
  if(!modemGSM.isGprsConnected())
    setupGSM();
  if (!MQTT.connected()) 
    reconnectMQTT();
}

//GSM 
void setupGSM() {
  Serial.println("Setup GSM...");
  modemGSM.init();

  while (!modemGSM.gprsConnect(APN, APN_USER, APN_KEY)) {
    Serial.println("GPRS Connection Failed");
    modemGSM.restart();
    delay(1000);
  }
  Serial.println("GPRS Connection Success");
}

void setup()
{
  Serial.begin(115200);
  SerialGSM.begin(9600);

  Serial.println();
  setupGSM();
  initMQTT();
}

void loop() {
  checkConnections();
  MQTT.loop();

  delay(2000);
}

Scenario, steps to reproduce

Use a sim800l with a Oi sim

Expected result

Setup GSM...
GPRS Connection Success
Connected to the MQTT broker

Actual result

I would like to connect to a mqtt broker using port 8883, but I cannot configure the SSL certificate.

Setup GSM...
GPRS Connection Success
* Trying to connect to the MQTT Broker: <broker_url>
Failed to reconnect to the broker.
Status: -2

I tried to use this example but it was unable to perform any AT commands and it always returned. I changed to the SoftwareSerial library but that scenario has not changed.

So I tried to configure the certificate manually using the AT commands

AT+FSCREATE=C:\CERT.CRT

OK

AT+FSWRITE=C:\CERT.CRT,0,1201,10
>
-----BEGIN CERTIFICATE-----
**********************************
-----END CERTIFICATE-----
OK

AT+SSLSETCERT=C:\CERT.CRT\

OK

+SSLSETCERT: 1

Debug and AT command log

AT

OK
AT&W

OK
AT+CFUN=0

OK
AT+CFUN=1,1

OK
[15756] ### TinyGSM Version: 0.10.5
[15756] ### TinyGSM Compiled Module:  TinyGsmClientSIM800
AT

OK
ATE0
ATE0

OK
AT+CMEE=2

OK
AT+GMM

SIMCOM_SIM800L

OK
[16074] ### Modem: SIMCOM SIM800L
[16074] ### Modem: SIMCOM SIM800L
AT+CLTS=1

OK
AT+CBATCHK=1

OK
AT+CPIN?

+CME ERROR: CFUN state is 0 or 4
AT+CPIN?

+CPIN: READY

OK
ATI

SIM800 R14.18

OK
AT+CPIN?

+CPIN: READY

OK
AT+CREG?

+CREG: 0,2

OK

**And CREG stays that way forever

What is the correct way to connect to an MQTT broker using SSL? What can I be doing wrong?

Mr-HaleYa commented 3 years ago

what country are you located in? what sim card are you using?

llaneiro commented 3 years ago

@Mr-HaleYa Brazil. I am currently using the SIM of the Brazilian operator Oi I intend to switch to a Things Mobile IoT SIM, but it has not arrived yet

better-adapted commented 3 years ago

@llaneiro

is it the same issue as mine? https://github.com/vshymanskyy/TinyGSM/issues/428

I did not have to load certs but enable the ssl option before attempting connections

see my fork for simple secure mqtt example

llaneiro commented 3 years ago

@llaneiro

is it the same issue as mine?

428

I did not have to load certs but enable the ssl option before attempting connections

see my fork for simple secure mqtt example

Unfortunately it didn't work for me I added this command to my setupGSM function, and he was successfully executed

AT+SSLOPT=1,1

OK

But when trying to connect to the broker still fails

* Trying to connect to the MQTT Broker: <broker_url>
Failed to reconnect to the broker.
Status: -4

These are the results of AT commands using the Stream Debugger

* Trying to connect to the MQTT Broker: <broker_url>
AT+CIPCLOSE=0,1

+CME ERROR: operation not allowed
AT+CIPSSL=0

OK
AT+CIPSTART=0,"TCP","<broker_url>",8883

OK

0, CONNECT OK
AT+CIPSEND=0,47

>-MQTT� <client_ID> <broker_user><broker_password>
DATA ACCEPT:0,47
AT+CIPRXGET=4,0

+CIPRXGET: 4,0,0

OK
AT+CIPSTATUS=0

+CIPSTATUS: 0,0,"TCP","3.13.162.221","8883","CONNECTED"

OK
AT+CIPRXGET=4,0

+CIPRXGET: 4,0,0

AT+CIPSTATUS=0

+CIPSTATUS: 0,0,"TCP","3.13.162.221","8883","REMOTE CLOSING"

OK
AT+CIPRXGET=4,0

+CIPRXGET: 4,0,0
better-adapted commented 3 years ago

AT+CIPSSL=0 is an issue

does not look like you are using WiFiClientSecure client - I followed all the code to find out the same thing see my fork for secure mqtt example

include

WiFiClientSecure client; instead of just WiFiClient client;

llaneiro commented 3 years ago

@better-adapted did you mean TinyGsmClient Secure? If so, you're right, I was doing some tests and forgot to go back to TinyGsmClientSecure

I switched to Secure Client and tried again. That was the result

* Trying to connect to the MQTT Broker: <broker_url>
AT+CIPCLOSE=0,1

+CME ERROR: operation not allowed
AT+CIPSSL=1

OK
AT+CIPSTART=0,"TCP","<broker_url>",8883

OK

0, CLOSE OK
Failed to reconnect to the broker.
better-adapted commented 3 years ago

Sorry but a few basic questions

Have you done AT+SSLOPT=1,1 just after modem.restart()?

Have you tried another broker like cloudmqtt? you can test non secure and secure there - beware port numbers

What firmware version is on the module? (I have SIM800 R14.18 on a legit SIM800L)

llaneiro commented 3 years ago

Have you done AT+SSLOPT=1,1 just after modem.restart()?

Yes, but i'm using modem.init(), some problem?

void setupGSM() {
  Serial.println("Setup GSM...");
  modemGSM.init();

   modemGSM.sendAT("+SSLOPT=1,1");
   if (modemGSM.waitResponse() != 1) {
     Serial.printf("modem +SSLOPT=1,1 failed");
   }
}

Have you tried another broker like cloudmqtt? you can test non secure and secure there - beware port numbers

Yes, but only without SSL. I tested with a broker maqiatto.com on port 1883 and it works perfectly.

What firmware version is on the module? (I have SIM800 R14.18 on a legit SIM800L)

SIM800 R14.18 on a legit SIM800L too

better-adapted commented 3 years ago

I would try the cloud mqtt broker on a secure port as well just to see (its free and handy!)

originally it all worked for me on cloudmqtt - until I got to Azure where I had to have "+SSLOPT=1,1" as well for certicate reasons

check the doc's on the +SSLOPT=1,1 command too

llaneiro commented 3 years ago

Is it still possible to create a free instance on it? I tried but there is a warning written "Plan Cute Cat is no longer available." Searching further I found this. Apparently it is no longer possible to create. 😔

SRGDamia1 commented 3 years ago

@llaneiro - this looks like yet another replicate of #418 and #378 and the truth is that I have no idea how to fix it and I don't even have the ability to test anything because I have neither a SIM800 that supports SSL nor access to a decent workspace in a location where 2G service is available.

llaneiro commented 3 years ago

@SRGDamia1 It is very similar. Apparently many people have problems using SSL with this module. I've also been trying for weeks and it doesn't work at all.

genotix commented 3 years ago

It would be of great value if we could get the TinyGSM module working with MQTTS. So far it seems that TLS v1.2 remains -unsupported- though the ESP32 is able to "talk" TLS 1.2 I have been thinking perhaps the TLS 1.2 activities could perhaps be offloaded to the ESP32?

Development activity on the SIM800 seem to be minimal though. Would appreciate to know which ESP32 compatible Modem devices could work with AWS and thus MQTTS.

SRGDamia1 commented 3 years ago

@genotix - Did you look at the list on the ReadMe for modem that support SSL? I think, of those supported, it's only the SIM800 that frequently gives people headaches. At least, that's what I see a lot of issues here for. As I mentioned before, I'm really not able to do any testing on anything 2G where I'm located and I don't currently have a use for SSL so I'm not able to give it time right now.

genotix commented 3 years ago

Ok, I’m currently trying to do the SSLClient integration to be able to offload the TLS activity and work with a SIM800L on AWS. I hope it works!

On 22 Jan 2021, at 18:11, Sara Damiano notifications@github.com wrote:

@genotix https://github.com/genotix - Did you look at the list on the ReadMe for modem that support SSL? I think, of those supported, it's only the SIM800 that frequently gives people headaches. At least, that's what I see a lot of issues here for. As I mentioned before, I'm really not able to do any testing on anything 2G where I'm located and I don't currently have a use for SSL so I'm not able to give it time right now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vshymanskyy/TinyGSM/issues/425#issuecomment-765560699, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCG53QEQKWWSO2D7PSWPZDS3GWU7ANCNFSM4PIV4H4A.

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/vshymanskyy/TinyGSM/issues/425#issuecomment-765560699", "url": "https://github.com/vshymanskyy/TinyGSM/issues/425#issuecomment-765560699", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

punkman commented 3 years ago

I have a LilyGo board that mounts a SIM800C modem (R14.18) and like others I cannot connect to my MQTT server using SSL (a self hosted Mosquitto).

Seems that the problems lies into the SIM800C that supports only TLS1.0. This is described into the document SIM800 Series_SSL_Application Note_V1.02

I tried to set Mosquitto to accept TLSv1 with tls_version tlsv1 in mosquitto.conf but regardless of this setting the connection was still refused. My distro is Debian 10 Buster and OpenSSL is set to refuse every connection that use anything lower than TLSv1.2.

When I lowered the system default setting of OpenSSL in /etc/ssl/openssl.cnf I successufully connected to my MQTT server using SSL.

[system_default_sect]
MinProtocol = TLSv1 #From TLSv1.2

Without a firmware update of the SIM800C modem or a way to offload SSL TLSv1.2 handshake to ESP32 I think we need to accept to use a lower TLS version with his security implications. I'm not a big fan of this solution...but I cannot find something better!

Now I'm trying to use ArduinoBearSSL client over a regular TinyGsmClient to handle SSL into ESP32 but without much success...

punkman commented 3 years ago

I made some progresses using SSLClient library to connect to MQTT server using TLSv1.2.

In broad terms:

Here is a minimal example (check placeholders):

#define SIM800C_AXP192_VERSION_20200609
#define TINY_GSM_MODEM_SIM800
#define TINY_GSM_RX_BUFFER 1024
#define TINY_GSM_USE_GPRS true

#include "utilities.h" //Some data for my specific board LILYGO TTGO T-Call&SIM800C
#include <SSLClient.h>
#include "trust_anchors.h" //Generated TA header, must be included after SSLClient
#include <TinyGsmClient.h>
#include <PubSubClient.h>

// Your GPRS credentials (leave empty, if missing)
const char apn[]      = "<YOUR_APN>"; // Your APN
const char gprsUser[] = ""; // User, if any
const char gprsPass[] = ""; // Password, if any
const char simPIN[]   = ""; // SIM card PIN code, if any

// MQTT Server credentials
#define CLIENT_NAME "<CLIENT_NAME>"
#define MQTT_HOST "<MQTT_URL_IP>"
#define MQTT_USER "<MQTT_USER>"
#define MQTT_PASS "<MQTT_PASS>"

long lastUpdate = -1; //Used to limit publish topic every 10s

TinyGsm modem(Serial1);
TinyGsmClient tinyGsmClient(modem);
SSLClient sslClient(tinyGsmClient, TAs, (size_t)TAs_NUM, 34);
    //TAs are the trust anchors defined in the header
    //34 is an analog pin, for Arduino analog pin is usually A5 
PubSubClient mqttClient(sslClient);

void reconnect_mqtt(){
  while (!mqttClient.connected()){
    if(mqttClient.connect(CLIENT_NAME, MQTT_USER, MQTT_PASS, "tests/"CLIENT_NAME"/status", 0, true, "OFFLINE")){
      mqttClient.publish("tests/"CLIENT_NAME"/status", "ONLINE", true);
    }
    else{
      delay(5000);
    }
  }
}

void setupModem(){
  //Init modem chip specific of this board
  pinMode(MODEM_PWRKEY, OUTPUT);
  pinMode(MODEM_POWER_ON, OUTPUT);
  digitalWrite(MODEM_POWER_ON, HIGH);
  digitalWrite(MODEM_PWRKEY, HIGH);
  delay(100);
  digitalWrite(MODEM_PWRKEY, LOW);
  delay(1000);
  digitalWrite(MODEM_PWRKEY, HIGH);
  delay(1000);
  // END init modem

  modem.restart();

  while(!modem.waitForNetwork(240000L)){
    delay(10000);
  }

  while(!modem.gprsConnect(apn, gprsUser, gprsPass)){
    delay(10000);
  }
}

void setup(){
  Serial1.begin(115200, SERIAL_8N1, MODEM_RX, MODEM_TX);
  delay(500);

  //Setup MQTT
  mqttClient.setServer(MQTT_HOST, 8883);
  mqttClient.setKeepAlive(20);
}

void loop(){
  if (!modem.isGprsConnected()) {
    setupModem();
  }

  if(!mqttClient.loop()){
    reconnect_mqtt();
  }

  long now = millis();
  if (mqttClient.connected() && (now-lastUpdate > 10000)) {
    lastUpdate = now;
    char s[20];
    sprintf(s, "%ul", millis());
    mqttClient.publish("tests/"CLIENT_NAME"/data", s);
  }
}
saukothari commented 3 years ago

@punkman , Thank you. I tried this approach with AWS IOT, and combined your approach of TinyGSM and SSLClient with client certificates. This worked to an extent that i could send the data on mqtt to AWS. But then I send the data from AWS to the device it gives read buffer discarded in favor of write. Not sure if the problem now is with SSLClient or the PubSubClient.

adamalfath commented 3 years ago

Hi, I've notice on the latest SSL App Note v1.05 it mentioned that SIM800 module has support up to TLS 1.2 but I don't know which firmware did they based on this documentation. Mine is 1418B05SIM868M32_BT (Model SIM868) but still cannot get the CONNECT OK from CIPSTART command using TLS 1.2.

Screenshot_8

pablofr918 commented 2 years ago

Hello @punkman, I tried your sketch, but got a lot of SSL errors, which may be caused because I don't understand the process. I'm using:

ESP32 SIM800C

With AT commands, I was able to connect to a SSL broker with a TCP connection, so that tells me it is possible, but when I try it with TinyGSM, I can't make it. In my case I want to connect to a test.mosquitto.org broker with port:8883 to be secured. I have tried to make the connection through PubSubclient and TinyGSMClientSecure, but no luck, so I'm trying the way you told, wraping the TinyGSMClient in SSLClient.

So the steps to follow are the following? if I want to connect to test.mosquitto.org, first I have to create the trust anchors with the page you pasted. Then I have to copy the result on the trust_anchors.h file. Run the code changing the apn and the broker.

Doing this, I get the following:

SSLClient)(SSL_WARN)(m_run_until): Terminating because the ssl engine closed 09:44:25.866 -> (SSLClient)(SSL_ERROR)(m_start_ssl): Failed to initlalize the SSL layer 09:44:25.866 -> (SSLClient)(SSL_ERROR)(m_print_br_error): Chain could not be linked to a trust anchor. See https://github.com/OPEnSLab-OSU/SSLClient/blob/master/TrustAnchors.md 09:44:30.868 -> (SSLClient)(SSL_WARN)(connect): Arduino client is already connected? Continuing anyway...

Do you know anything about this? Thank you!!