search

arendst / Tasmota

Alternative firmware for ESP8266 and ESP32 based devices with easy configuration using webUI, OTA updates, automation using timers or rules, expandability and entirely local control over MQTT, HTTP, Serial or KNX. Full documentation at
https://tasmota.github.io/docs
GNU General Public License v3.0
22.17k stars 4.8k forks source link

Telegram message notification support #8619

Closed ademirci closed 4 years ago

ademirci commented 4 years ago

Have you looked for this feature in other issues and in the docs?
Thank you for this great software. Yes, I searched in the repsitory and can not see related result. Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. No, it is not a problem Describe the solution you'd like
A clear and concise description of what you want to happen. I want to send a notification as Telegram message so that I can recieve to my phone anywhere. Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered. There are IOT MQTT alternatives for notification but they are not straightforward. Additional context
Add any other context or screenshots about the feature request here. I noticed following arduino library in the github and maybe it is easy to integrate it to Tasmota https://github.com/witnessmenow/Universal-Arduino-Telegram-Bot

(Please, remember to close the issue when the problem has been addressed)

dzatoah commented 4 years ago

That would be Awesome! I can imagine a command like notify "..." for rules or sensor values!!

kugelkopf123 commented 4 years ago

https://github.com/ArduinoHannover/Pushover Another one that works.

ascillato commented 4 years ago

Duplicated https://github.com/arendst/Tasmota/issues/1456

arendst commented 4 years ago

Been working on this yesterday but failed to get a https connection to api.telegram.org

All mentionend libraries fail in this case as is also reported by their different issues.

Tried a very basic approach using all SSL provided libraries but still fail to make a connection.

If anyone manages to make a https connection using the current core version I'll be interessted to dig in it again.

s-hadinger commented 4 years ago

Theo, I know very well the SSL part. I can try.

arendst commented 4 years ago

This is the driver code currently not using any library other than the available BearSSL.

xdrv_50_telegram.zip

arendst commented 4 years ago

You'll need this too

tasmota.zip

arendst commented 4 years ago

And define USE_TELEGRAM in user_config_overrid.h

ademirci commented 4 years ago

I am excited to see the result. Even we can send special format messages to Tasmota to enable or disable functions or rules over internet without the need for an IOT account. Also we can collect some sensor values periodically with another Telegram bot and setup our own IOT dashboard for free.

eku commented 4 years ago

@ademirci well a plain XMPP/Jabber client would be enougth to reach public or private XMPP/Jabber chat server. And, furthermore, the desired function could be performed by the MQTT servers instead of overloading the client. @arendst I would rather invest the time in support for an open federal platform than support a proprietary solution like Telegram.

s-hadinger commented 4 years ago

Telegram api is HTTP not XMPP which makes it far easier to integrate in Tasmota.

What do you mean by "open federal platform"?

eku commented 4 years ago

@s-hadinger wrote:

What do you mean by "open federal platform"?

Anyone can run a server and you do not need to have an account on the same server where the chat partner has one. Classic examples are Email and XMPP/Jabber, but also Mastodon.

s-hadinger commented 4 years ago

At first glance, api.telegram.org uses a certificate signed by GoDaddy Daddy Secure Certificate Authority - G2. The server key is 2048 bits and the proposed cipher is ECDHE-RSA-AES128-GCM-SHA256 which is exactly the cipher used with AWS IoT.

This is all good news, apart from adding the CA in Flash, it should run smoothly. Stay tuned.

If you're curious about this, try openssl s_client -showcerts -connect api.telegram.org:443

arendst commented 4 years ago

Just did. Nice find!

s-hadinger commented 4 years ago

And here is the CA ready to be munched by BearSSL (added at the end of tasmota_ca.ino:

#ifdef USE_TELEGRAM
// we add a separate CA for telegram

/*********************************************************************************************\
 * GoDaddy Daddy Secure Certificate Authority - G2, RSA 2048 bits SHA 256, valid until 20220523
 *
 * to convert do: "brssl ta GoDaddyCA.pem"
 * then copy and paste below, chain the generic names to the same as below
 * remove "static" and add "PROGMEM"
\*********************************************************************************************/

const unsigned char GoDaddyCAG2_DN[] PROGMEM = {
    0x30, 0x3E, 0x31, 0x21, 0x30, 0x1F, 0x06, 0x03, 0x55, 0x04, 0x0B, 0x13,
    0x18, 0x44, 0x6F, 0x6D, 0x61, 0x69, 0x6E, 0x20, 0x43, 0x6F, 0x6E, 0x74,
    0x72, 0x6F, 0x6C, 0x20, 0x56, 0x61, 0x6C, 0x69, 0x64, 0x61, 0x74, 0x65,
    0x64, 0x31, 0x19, 0x30, 0x17, 0x06, 0x03, 0x55, 0x04, 0x03, 0x13, 0x10,
    0x61, 0x70, 0x69, 0x2E, 0x74, 0x65, 0x6C, 0x65, 0x67, 0x72, 0x61, 0x6D,
    0x2E, 0x6F, 0x72, 0x67
};

const unsigned char GoDaddyCAG2_RSA_N[] PROGMEM = {
    0xB4, 0xA3, 0x16, 0x9E, 0x5C, 0x57, 0xC9, 0x89, 0x65, 0xED, 0xEA, 0x78,
    0x0B, 0xAE, 0x8A, 0x58, 0x2F, 0xAE, 0x5A, 0xC8, 0x6E, 0x49, 0x8D, 0xFC,
    0x57, 0xA5, 0x98, 0x88, 0x78, 0x2E, 0x0B, 0x3C, 0x40, 0x3C, 0x21, 0x2E,
    0x9A, 0x94, 0x98, 0x33, 0xA7, 0xE3, 0x42, 0xA7, 0x85, 0xFA, 0xD0, 0x73,
    0x84, 0x01, 0x1C, 0x72, 0x39, 0x37, 0x23, 0xB5, 0x56, 0x1D, 0x43, 0xA5,
    0x71, 0x14, 0x08, 0x24, 0xA5, 0x39, 0xCC, 0xDE, 0x58, 0x53, 0x94, 0x8E,
    0x2A, 0x42, 0xA7, 0x4E, 0x2D, 0x07, 0x32, 0x9E, 0xBA, 0x8B, 0xD3, 0x2A,
    0xA9, 0x9E, 0xC0, 0xE3, 0xCE, 0x9A, 0x10, 0x96, 0x45, 0x58, 0x7A, 0xC7,
    0x1E, 0x45, 0x14, 0x23, 0x92, 0xBB, 0x54, 0x82, 0x88, 0x94, 0x49, 0xB6,
    0xBE, 0x81, 0x21, 0x00, 0x29, 0x6D, 0xC9, 0xCE, 0x8B, 0x39, 0x3A, 0xDC,
    0x35, 0x15, 0xD9, 0xEB, 0x47, 0x9C, 0xEF, 0xBA, 0x09, 0x0E, 0x16, 0xE4,
    0xD9, 0xEB, 0x72, 0x30, 0xFA, 0x49, 0xAB, 0x98, 0x31, 0x7C, 0xB3, 0xAC,
    0x2B, 0x29, 0x91, 0x87, 0x08, 0x41, 0x72, 0x5E, 0x35, 0xC7, 0x87, 0x04,
    0x22, 0xF5, 0x48, 0x76, 0x30, 0x6D, 0x88, 0xDF, 0xF2, 0xA5, 0x29, 0x13,
    0x70, 0xB3, 0x87, 0x02, 0xD5, 0x6B, 0x58, 0xB1, 0xE8, 0x73, 0xC7, 0xE4,
    0xEF, 0x79, 0x86, 0xA4, 0x07, 0x5F, 0x67, 0xB4, 0x79, 0x8D, 0xA4, 0x25,
    0x01, 0x82, 0x8C, 0xE0, 0x30, 0x17, 0xCB, 0x4B, 0x5C, 0xFB, 0xEB, 0x4C,
    0x12, 0x51, 0xB9, 0xC9, 0x04, 0x1F, 0x7E, 0xD2, 0xF8, 0xBA, 0xF5, 0x35,
    0x8D, 0x8A, 0x1C, 0x37, 0x82, 0xF0, 0x15, 0x73, 0x00, 0x6E, 0x3D, 0x1C,
    0x76, 0x8B, 0x01, 0x74, 0x81, 0x3D, 0xE4, 0x2C, 0xA7, 0xCC, 0x2F, 0x66,
    0xDC, 0x44, 0xA8, 0x27, 0x3F, 0xEA, 0xD0, 0xA7, 0xA8, 0xF1, 0xCB, 0xEA,
    0xDA, 0x07, 0x38, 0xBD
};

const unsigned char GoDaddyCAG2_RSA_E[] PROGMEM = {
    0x01, 0x00, 0x01
};

const br_x509_trust_anchor GoDaddyCAG2_TA[1] PROGMEM = {
    {
        { (unsigned char *)TA0_DN, sizeof TA0_DN },
        0,
        {
            BR_KEYTYPE_RSA,
            { .rsa = {
                (unsigned char *)GoDaddyCAG2_RSA_N, sizeof GoDaddyCAG2_RSA_N,
                (unsigned char *)GoDaddyCAG2_RSA_E, sizeof GoDaddyCAG2_RSA_E,
            } }
        }
    }
};

#endif // USE_TELEGRAM

The downside is that the certificate is valid until May 2022, so we will need to update regularly.

s-hadinger commented 4 years ago

I will not be available this afternoon to test the whole, but here is how it should work:

#include "WiFiClientSecureLightBearSSL.h"
BearSSL::WiFiClientSecure_light *tlsClient;

[...]
  tlsClient = new BearSSL::WiFiClientSecure_light(1024,1024);
  tlsClient->setTrustAnchor(&GoDaddyCAG2_TA);

Then pass tlsClient to the Telegram lib.

Edit: it would be interesting to print tlsClient->getMFLNStatus() after the connection to see if we risk dropping messages of more than 1024 bytes.

If connection failed, tlsClient->getLastError() will give the SSL error number.

arendst commented 4 years ago

Thx I'll give it a try today

s-hadinger commented 4 years ago

Oh I forgot, you need to set #define USE_MQTT_TLS_FORCE_EC_CIPHER to force ECDHE-RSA-AES128-GCM-SHA256

s-hadinger commented 4 years ago

I'm trying now, actually here is the CA:

#ifdef USE_TELEGRAM
// we add a separate CA for telegram

/*********************************************************************************************\
 * GoDaddy Daddy Secure Certificate Authority - G2, RSA 2048 bits SHA 256, valid until 20220523
 *
 * to convert do: "brssl ta GoDaddyCA.pem"
 * then copy and paste below, chain the generic names to the same as below
 * remove "static" and add "PROGMEM"
\*********************************************************************************************/

const unsigned char GoDaddyCAG2_DN[] PROGMEM = {
    0x30, 0x3E, 0x31, 0x21, 0x30, 0x1F, 0x06, 0x03, 0x55, 0x04, 0x0B, 0x13,
    0x18, 0x44, 0x6F, 0x6D, 0x61, 0x69, 0x6E, 0x20, 0x43, 0x6F, 0x6E, 0x74,
    0x72, 0x6F, 0x6C, 0x20, 0x56, 0x61, 0x6C, 0x69, 0x64, 0x61, 0x74, 0x65,
    0x64, 0x31, 0x19, 0x30, 0x17, 0x06, 0x03, 0x55, 0x04, 0x03, 0x13, 0x10,
    0x61, 0x70, 0x69, 0x2E, 0x74, 0x65, 0x6C, 0x65, 0x67, 0x72, 0x61, 0x6D,
    0x2E, 0x6F, 0x72, 0x67
};

const unsigned char GoDaddyCAG2_RSA_N[] PROGMEM = {
    0xB4, 0xA3, 0x16, 0x9E, 0x5C, 0x57, 0xC9, 0x89, 0x65, 0xED, 0xEA, 0x78,
    0x0B, 0xAE, 0x8A, 0x58, 0x2F, 0xAE, 0x5A, 0xC8, 0x6E, 0x49, 0x8D, 0xFC,
    0x57, 0xA5, 0x98, 0x88, 0x78, 0x2E, 0x0B, 0x3C, 0x40, 0x3C, 0x21, 0x2E,
    0x9A, 0x94, 0x98, 0x33, 0xA7, 0xE3, 0x42, 0xA7, 0x85, 0xFA, 0xD0, 0x73,
    0x84, 0x01, 0x1C, 0x72, 0x39, 0x37, 0x23, 0xB5, 0x56, 0x1D, 0x43, 0xA5,
    0x71, 0x14, 0x08, 0x24, 0xA5, 0x39, 0xCC, 0xDE, 0x58, 0x53, 0x94, 0x8E,
    0x2A, 0x42, 0xA7, 0x4E, 0x2D, 0x07, 0x32, 0x9E, 0xBA, 0x8B, 0xD3, 0x2A,
    0xA9, 0x9E, 0xC0, 0xE3, 0xCE, 0x9A, 0x10, 0x96, 0x45, 0x58, 0x7A, 0xC7,
    0x1E, 0x45, 0x14, 0x23, 0x92, 0xBB, 0x54, 0x82, 0x88, 0x94, 0x49, 0xB6,
    0xBE, 0x81, 0x21, 0x00, 0x29, 0x6D, 0xC9, 0xCE, 0x8B, 0x39, 0x3A, 0xDC,
    0x35, 0x15, 0xD9, 0xEB, 0x47, 0x9C, 0xEF, 0xBA, 0x09, 0x0E, 0x16, 0xE4,
    0xD9, 0xEB, 0x72, 0x30, 0xFA, 0x49, 0xAB, 0x98, 0x31, 0x7C, 0xB3, 0xAC,
    0x2B, 0x29, 0x91, 0x87, 0x08, 0x41, 0x72, 0x5E, 0x35, 0xC7, 0x87, 0x04,
    0x22, 0xF5, 0x48, 0x76, 0x30, 0x6D, 0x88, 0xDF, 0xF2, 0xA5, 0x29, 0x13,
    0x70, 0xB3, 0x87, 0x02, 0xD5, 0x6B, 0x58, 0xB1, 0xE8, 0x73, 0xC7, 0xE4,
    0xEF, 0x79, 0x86, 0xA4, 0x07, 0x5F, 0x67, 0xB4, 0x79, 0x8D, 0xA4, 0x25,
    0x01, 0x82, 0x8C, 0xE0, 0x30, 0x17, 0xCB, 0x4B, 0x5C, 0xFB, 0xEB, 0x4C,
    0x12, 0x51, 0xB9, 0xC9, 0x04, 0x1F, 0x7E, 0xD2, 0xF8, 0xBA, 0xF5, 0x35,
    0x8D, 0x8A, 0x1C, 0x37, 0x82, 0xF0, 0x15, 0x73, 0x00, 0x6E, 0x3D, 0x1C,
    0x76, 0x8B, 0x01, 0x74, 0x81, 0x3D, 0xE4, 0x2C, 0xA7, 0xCC, 0x2F, 0x66,
    0xDC, 0x44, 0xA8, 0x27, 0x3F, 0xEA, 0xD0, 0xA7, 0xA8, 0xF1, 0xCB, 0xEA,
    0xDA, 0x07, 0x38, 0xBD
};

const unsigned char GoDaddyCAG2_RSA_E[] PROGMEM = {
    0x01, 0x00, 0x01
};

const br_x509_trust_anchor GoDaddyCAG2_TA PROGMEM = {
    { (unsigned char *)GoDaddyCAG2_DN, sizeof GoDaddyCAG2_DN },
    0,
    {
        BR_KEYTYPE_RSA,
        { .rsa = {
            (unsigned char *)GoDaddyCAG2_RSA_N, sizeof GoDaddyCAG2_RSA_N,
            (unsigned char *)GoDaddyCAG2_RSA_E, sizeof GoDaddyCAG2_RSA_E,
        } }
    }
};

#endif // USE_TELEGRAM
s-hadinger commented 4 years ago

I'm progressing. Call is done, I'm now getting error 296 which is SSL3_ALERT_HANDSHAKE_FAILURE

I will continue tomorrow. The current code is here: https://github.com/s-hadinger/Tasmota/tree/telegram_dev

s-hadinger commented 4 years ago

I got a first version working by disabling CA validation, otherwise it seems to hang.

First good news is TLS was indeed connected, second good news is that Telegram supports MFLN so we can keep 1KB buffers. Bad news is it crashed in the destructor of HTTPClient, but I know why.

Fun fact: after doing lots of tests by IP address got black-listed. I needed to switch to mobile.

GET Update Messages
Client initialized
connect(api.telegram.org,443)
connect(149.154.167.220,443)
_connectSSL: start connection
Connected! MFLNStatus = 1
[HTTPS] GET... code: 404

Exception (28):
epc1=0x4023a0e7 epc2=0x00000000 epc3=0x40248b9d excvaddr=0x000002c0 depc=0x00000000

>>>stack>>>

ctx: cont
sp: 3ffffc40 end: 3fffffc0 offset: 0190
3ffffdd0:  00000001 3fff2fb8 3fff4b84 40208e70
s-hadinger commented 4 years ago

Crash fixed. It seems to connect correctly, since I get a 404 error because of a fake bot token.

Theo, the code shouldn't retry automatically when there is an error, because it ends up hammering the api endpoint and makes the ip address black-listed.

Log is very verbose for now. You need for now to #define USE_TELEGRAM and do not define any TLS option, especially don't enable USE_MQTT_TLS_CA_CERT (for now)

s-hadinger commented 4 years ago

Progressing. Connection works well and I need to clean up code.

However the current logic reconnects every second, which is problematic: the actual SSL connection takes around 1 second and freezes the entire Tasmota. Is long polling possible? Or should me relax the frequency?

Also send does not seem to work (yet).

connect(api.telegram.org,443)
connect(149.154.167.220,443)
_connectSSL: start connection
Connected! MFLNStatus = 1
[HTTPS] GET... code: 200
[HTTPS] Received payload telegram:
1BTC = {"ok":true,"result":[]}USD
Sent Update request messages up to : 0
no new messages
s-hadinger commented 4 years ago

Hi have a dilemma, for TLS server validation:

  1. validate the CA. Problem: the server certificate is bigger than 1024 bytes, so I need to increase the receive buffer from 1024 to 1536 bytes and eats up 512 more byte or RAM. Also, the certificate is only valid until 2022.
  2. is fingerprint validation of server public key. I have no clue about the stability of their public key, it may break tomorrow or stay stable for 10 years.
  3. don't check the server identity which is subject to man-in-the-middle attack.

I propose to implement both 1 and 2, it will depend on #define USE_MQTT_TLS_CA_CERT value.

s-hadinger commented 4 years ago

Done. The TLS is fully working, both with CA validation or Fingerprint validation. If the fingerprint is wrong, the new one is shown in logs.

@arendst You can now work on the fun part :)

All needed changes are in this branch: https://github.com/s-hadinger/Tasmota/tree/telegram_dev

arendst commented 4 years ago

Thx for the basics. Will see tomorrow.

arendst commented 4 years ago

Just had a very quick look but failed to get anything running. Both my and your repository end in this:

00:00:03 WIF: Attempting connection...
00:00:04 WIF: Network 0, AP2, SSId indebuurt_IoT, Channel 1, BSSId 74:83:C2:2A:F1:AC, RSSI -85, Encryption 1
00:00:04 WIF: Network 1, AP2, SSId indebuurt_IoT, Channel 11, BSSId 18:E8:29:CA:17:C1, RSSI -23, Encryption 1
00:00:04 WIF: Connecting to AP2 indebuurt_IoT Channel 11 BSSId 18:E8:29:CA:17:C1 in mode 11N as wemos4...
00:00:04 WIF: Attempting connection...
00:00:05 WIF: Connected
00:00:05 HTP: Web server active on wemos4 with IP address 192.168.2.223
18:25:27 NTP: UTC 2020-06-08T16:25:26, DST 2020-03-29T02:00:00, STD 2020-10-25T03:00:00
18:25:27 TLG: Started
18:25:28 MQT: Attempting connection...
18:25:28 MQT: Connected
18:25:28 MQT: tele/wemos4/LWT = Online (retained)
18:25:28 MQT: cmnd/wemos4/POWER = 
18:25:28 MQT: Subscribe to cmnd/wemos4/#
18:25:28 MQT: Subscribe to cmnd/tasmotas/#
18:25:28 MQT: Subscribe to cmnd/DVES_83BB10_fb/#
18:25:28 MQT: tele/wemos4/INFO1 = {"Module":"Sonoff Basic","Version":"8.3.1.2(tasmota)","FallbackTopic":"cmnd/DVES_83BB10_fb/","GroupTopic":"cmnd/tasmotas/"}
18:25:28 MQT: tele/wemos4/INFO2 = {"WebServerMode":"Admin","Hostname":"wemos4","IPAddress":"192.168.2.223"}
18:25:28 MQT: tele/wemos4/INFO3 = {"RestartReason":{"Exception":4,"Reason":"Software Watchdog","EPC":["40003b53","00000000","00000000"],"EXCVADDR":"00000000","DEPC":"00000000"}}
18:25:28 MQT: stat/wemos4/RESULT = {"POWER":"OFF"}
18:25:28 MQT: stat/wemos4/POWER = OFF
GET Update Messages 
18:25:28 TLG: Cmnd bot1179906608:AAE-3UfPJt3u8d-gbGXslnFO96o3DahG6EM/getUpdates?offset=0

User exception (panic/abort/assert)
Abort called

>>>stack>>>

ctx: sys
sp: 3fff7ee0 end: 3fffffb0 offset: 0000
3fff7ee0:  f489b847 deadbeef 00000004 3fff6630  
3fff7ef0:  000000fe 00000000 00000000 00000000  
3fff7f00:  00000000 00000000 00000000 3fff7f30  
3fff7f10:  00000004 00000090 00000004 f489b847  
3fff7f20:  00000008 00000000 3fff5e4c 4024f63a  
3fff7f30:  3ffe892a 00000000 00000000 4024f64c  
3fff7f40:  3fff7c60 00000000 3fff5e4c 402085fd  
3fff7f50:  deadbeef 00000000 3fff5e4c 4020868c  
3fff7f60:  deadbeef 00000000 068508d8 40283e8c  
3fff7f70:  00000001 3fff54b8 40284128 40284128  
3fff7f80:  00000000 00000001 3fff81dc 00000100  
3fff7f90:  3fff82dc 00000003 3fff7fa4 3fff803c  
3fff7fa0:  4028d528 ffffffff 00000000 00000000  
3fff7fb0:  00000000 000000c2 00000000 0000001f  
3fff7fc0:  0000001f 000000ae 00000000 0000001f

No worries. Will have a better look tomorrow.

s-hadinger commented 4 years ago

You can try in WiFiClientSecureLightBearSSL.cpp:

#define DEBUG_TLS
#define DEBUG_ESP_SSL

Also try with and without #define USE_MQTT_TLS_CA_CERT

arendst commented 4 years ago

Frustrating but all I get is this:

00:00:01 HTP: Web server active on wemos4 with IP address 192.168.2.223
15:13:26 NTP: UTC 2020-06-09T13:13:25, DST 2020-03-29T02:00:00, STD 2020-10-25T03:00:00
15:13:26 TLG: Started
15:13:27 MQT: Attempting connection...
15:13:27 MQT: Connected
15:13:27 MQT: tele/wemos4/LWT = Online (retained)
15:13:27 MQT: cmnd/wemos4/POWER = 
15:13:27 MQT: Subscribe to cmnd/wemos4/#
15:13:27 MQT: Subscribe to cmnd/tasmotas/#
15:13:27 MQT: Subscribe to cmnd/DVES_83BB10_fb/#
15:13:27 MQT: tele/wemos4/INFO1 = {"Module":"Sonoff Basic","Version":"8.3.1.2(theo)","FallbackTopic":"cmnd/DVES_83BB10_fb/","GroupTopic":"cmnd/tasmotas/"}
15:13:27 MQT: tele/wemos4/INFO2 = {"WebServerMode":"Admin","Hostname":"wemos4","IPAddress":"192.168.2.223"}
15:13:27 MQT: tele/wemos4/INFO3 = {"RestartReason":"External System"}
15:13:27 MQT: stat/wemos4/RESULT = {"POWER":"OFF"}
15:13:27 MQT: stat/wemos4/POWER = OFF
15:13:31 MQT: tele/wemos4/STATE = {"Time":"2020-06-09T15:13:31","Uptime":"0T00:00:09","UptimeSec":9,"Heap":27,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"POWER":"OFF","Wifi":{"AP":2,"SSId":"indebuurt_IoT","BSSId":"18:E8:29:CA:17:C1","Channel":11,"RSSI":100,"Signal":-28,"LinkCount":1,"Downtime":"0T00:00:03"}}
15:13:32 APP: Boot Count 161
15:13:32 CFG: Saved to flash at F5, Count 290, Bytes 4096
GET Update Messages 
StackThunk before 27496, Fragmentation=2, Thunkstack=0, Free stack=3664, FreeContStack=1680
StackThunk after 27496, Fragmentation=2, Thunkstack=0, Free stack=3664, FreeContStack=1680
allocateBuffers before 27496, Fragmentation=2, Thunkstack=0, Free stack=3616, FreeContStack=1680
allocateBuffers ClientContext 24056, Fragmentation=2, Thunkstack=0, Free stack=3616, FreeContStack=1680
allocateBuffers after 21528, Fragmentation=2, Thunkstack=0, Free stack=3616, FreeContStack=1680
15:13:33 TLG: Cmnd bot1179906608:AAE-3UfPJt3u8d-gbGXslnFO96o3DahG6EM/getUpdates?offset=0
connect(api.telegram.org,443)
connect(149.154.167.220,443)
Before calling _connectSSL 21192, Fragmentation=1, Thunkstack=0, Free stack=3632, FreeContStack=1680
_connectSSL.start 21192, Fragmentation=1, Thunkstack=0, Free stack=3600, FreeContStack=1680
Thunk allocated 17584, Fragmentation=1, Thunkstack=0, Free stack=3600, FreeContStack=1680
_connectSSL: start connection
_connectSSL before DecoderContext allocation 17584, Fragmentation=1, Thunkstack=0, Free stack=3600, FreeContStack=1680
_connectSSL after DecoderContext allocation 16432, Fragmentation=1, Thunkstack=0, Free stack=3600, FreeContStack=1680
_connectSSL before PrivKey allocation 16432, Fragmentation=1, Thunkstack=0, Free stack=3600, FreeContStack=1680
_wait_for_handshake: start

User exception (panic/abort/assert)
Abort called

>>>stack>>>

ctx: sys
sp: 3fff7b38 end: 3fffffb0 offset: 0000
3fff7b38:  f2532668 deadbeef 00000004 3fff5e18  
3fff7b48:  000000fe 00000000 00000000 00000000  
3fff7b58:  00000000 00000000 00000000 3fff7b88

It seems it stalls in _run_until called from _wait_for_handshake but I don't see why.

s-hadinger commented 4 years ago

It seems to be a conditional compile issue. Get the latest commit to enable CA validation without a private key, and compile with both #define USE_MQTT_TLS_CA_CERT and #define USE_MQTT_TLS_FORCE_EC_CIPHER.

I'll check later how to make sure it compiles with the right options.

s-hadinger commented 4 years ago

I found the issue, the Stack Thunk was not correctly adjusted. Now the branch is good: https://github.com/s-hadinger/Tasmota/tree/telegram_dev

arendst commented 4 years ago

Continue testing ....

arendst commented 4 years ago

Progress using the attached xdrv_50_telegram.ino file

xdrv_50_telegram.zip

Still fails after three rounds but we will get there eventually...

18:16:54 CMD: tgstate 1
18:16:54 SRC: Serial
18:16:54 CMD: Group 0, Index 1, Command "TGSTATE", Data "1"
18:16:54 MQT: stat/wemos4/RESULT = {"TGState":{"Receive":"ON","Send":"OFF"}}
18:16:55 TLG: Started
18:16:56 TLG: Get update messages
StackThunk before 27392, Fragmentation=5, Thunkstack=0, Free stack=3648, FreeContStack=1788
StackThunk after 27392, Fragmentation=5, Thunkstack=0, Free stack=3648, FreeContStack=1788
allocateBuffers before 27392, Fragmentation=5, Thunkstack=0, Free stack=3600, FreeContStack=1788
allocateBuffers ClientContext 23952, Fragmentation=6, Thunkstack=0, Free stack=3600, FreeContStack=1788
allocateBuffers after 21424, Fragmentation=6, Thunkstack=0, Free stack=3600, FreeContStack=1788
18:16:56 TLG: Cmnd bot1179906608:AAE-3UfPJt3u8d-gbGXslnFO96o3DahG6EM/getUpdates?offset=0
connect(api.telegram.org,443)
connect(149.154.167.220,443)
Before calling _connectSSL 20440, Fragmentation=4, Thunkstack=0, Free stack=3616, FreeContStack=1788
_connectSSL.start 20440, Fragmentation=4, Thunkstack=0, Free stack=3584, FreeContStack=1788
Thunk allocated 15136, Fragmentation=5, Thunkstack=0, Free stack=3584, FreeContStack=1788
_connectSSL: start connection
_connectSSL before DecoderContext allocation 15136, Fragmentation=5, Thunkstack=0, Free stack=3584, FreeContStack=1788
_connectSSL after DecoderContext allocation 13984, Fragmentation=5, Thunkstack=0, Free stack=3584, FreeContStack=1788
_connectSSL before PrivKey allocation 13984, Fragmentation=5, Thunkstack=0, Free stack=3584, FreeContStack=1788
Connected! MFLNStatus = 1
_connectSSL.end 14096, Fragmentation=6, Thunkstack=4516, Free stack=3584, FreeContStack=1472
_connectSSL.end, freeing StackThunk 19400, Fragmentation=27, Thunkstack=0, Free stack=3584, FreeContStack=1472
_connectSSL after release of Priv Key 20552, Fragmentation=4, Thunkstack=0, Free stack=3584, FreeContStack=1472
18:16:57 TLG: Connected in 997 ms, max ThunkStack used 4516
_write 20464, Fragmentation=4, Thunkstack=0, Free stack=3568, FreeContStack=1472
_write 20464, Fragmentation=4, Thunkstack=0, Free stack=3552, FreeContStack=1472
18:16:57 TLG: Sent Update request messages up to 0
18:16:57 TLG: Msg 1 {"update_id":973125390,
"message":{"message_id":16,"from":{"id":139920293,"is_bot":false,"first_name":"Theo","last_name":"Arends","username":"arendst","language_code":"nl"},"chat":{"id":139920293,"first_name":"Theo","last_name":"Arends","username":"arendst","type":"private"},"date":1591805777,"text":"Tasmota is great"}}]}
18:16:57 TLG: Parsed0 "973125390"
18:16:57 TLG: Parsed1 "139920293"
18:16:57 TLG: Parsed2 "Theo"
18:16:57 TLG: Parsed3 "Arends"
18:16:57 TLG: Parsed4 "139920293"
18:16:57 TLG: Parsed5 "Tasmota is great"
18:16:57 TLG: Send message
StackThunk before 20488, Fragmentation=6, Thunkstack=0, Free stack=3696, FreeContStack=1472
StackThunk after 20488, Fragmentation=6, Thunkstack=0, Free stack=3696, FreeContStack=1472
allocateBuffers before 20488, Fragmentation=6, Thunkstack=0, Free stack=3648, FreeContStack=1472
allocateBuffers ClientContext 17048, Fragmentation=6, Thunkstack=0, Free stack=3648, FreeContStack=1472
allocateBuffers after 14520, Fragmentation=7, Thunkstack=0, Free stack=3648, FreeContStack=1472
18:16:57 TLG: Cmnd bot1179988888:AAE-3UfPJt3u8d-gbGXslnFAAAAAAahG6EM/sendMessage?chat_id=139920293&text=Tasmota is great&reply_markup=
connect(api.telegram.org,443)
connect(149.154.167.220,443)
Before calling _connectSSL 14184, Fragmentation=5, Thunkstack=0, Free stack=3664, FreeContStack=1472
_connectSSL.start 14184, Fragmentation=5, Thunkstack=0, Free stack=3632, FreeContStack=1472
Thunk allocated 8880, Fragmentation=8, Thunkstack=0, Free stack=3632, FreeContStack=1472
_connectSSL: start connection
_connectSSL before DecoderContext allocation 8880, Fragmentation=8, Thunkstack=0, Free stack=3632, FreeContStack=1472
_connectSSL after DecoderContext allocation 7728, Fragmentation=9, Thunkstack=0, Free stack=3632, FreeContStack=1472
_connectSSL before PrivKey allocation 7728, Fragmentation=9, Thunkstack=0, Free stack=3632, FreeContStack=1472
Connected! MFLNStatus = 1
_connectSSL.end 7840, Fragmentation=10, Thunkstack=4516, Free stack=3632, FreeContStack=1472
_connectSSL.end, freeing StackThunk 13144, Fragmentation=33, Thunkstack=0, Free stack=3632, FreeContStack=1472
_connectSSL after release of Priv Key 14296, Fragmentation=6, Thunkstack=0, Free stack=3632, FreeContStack=1472
18:16:58 TLG: Connected in 971 ms, max ThunkStack used 4516
_write 14160, Fragmentation=5, Thunkstack=0, Free stack=3616, FreeContStack=1472
_write 14160, Fragmentation=5, Thunkstack=0, Free stack=3600, FreeContStack=1472
18:16:58 TLG: Message sent
18:17:10 TLG: Get update messages
StackThunk before 14280, Fragmentation=6, Thunkstack=0, Free stack=3648, FreeContStack=1472
StackThunk after 14280, Fragmentation=6, Thunkstack=0, Free stack=3648, FreeContStack=1472
allocateBuffers before 14280, Fragmentation=6, Thunkstack=0, Free stack=3600, FreeContStack=1472
allocateBuffers ClientContext 10840, Fragmentation=7, Thunkstack=0, Free stack=3600, FreeContStack=1472
allocateBuffers after 8312, Fragmentation=8, Thunkstack=0, Free stack=3600, FreeContStack=1472
18:17:10 TLG: Cmnd bot1179988888:AAE-3UfPJt3u8d-gbGXslnFAAAAAAahG6EM/getUpdates?offset=973125391
connect(api.telegram.org,443)
connect(149.154.167.220,443)
Before calling _connectSSL 7976, Fragmentation=5, Thunkstack=0, Free stack=3616, FreeContStack=1472
_connectSSL.start 7976, Fragmentation=5, Thunkstack=0, Free stack=3584, FreeContStack=1472
Thunk allocated 2672, Fragmentation=13, Thunkstack=0, Free stack=3584, FreeContStack=1472
_connectSSL: start connection
_connectSSL before DecoderContext allocation 2672, Fragmentation=13, Thunkstack=0, Free stack=3584, FreeContStack=1472
_connectSSL after DecoderContext allocation 1520, Fragmentation=22, Thunkstack=0, Free stack=3584, FreeContStack=1472
_connectSSL before PrivKey allocation 1520, Fragmentation=22, Thunkstack=0, Free stack=3584, FreeContStack=1472

Exception (29):
epc1=0x4000df64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

>>>stack>>>

ctx: sys
sp: 3fffec10 end: 3fffffb0 offset: 0190
3fffeda0:  000000c1 3ffee860 00000002 40101210  
3fffedb0:  4025a553 3ffec5a0 00000000 4025a4e8  
3fffedc0:  00000002 4025a48f 00000002 402595e8  
3fffedd0:  40259611 3fffee80 3ffee860 00000016
s-hadinger commented 4 years ago

Good progress. There is a memory leak, your available memory dropped from 27KB to 8KB and crashed.

Edit: I don't have this memory leak in my version. Make sure to not reallocate a telegramClient object each time.

arendst commented 4 years ago

Initial support for Telegram messaging allows for sending commands to a Tasmota device and receiving response. It also allows sending (user)text from Tasmota to Telegram.

As comms between the two is slow (every message needs at least 1 second to complete connection and transfer) it's more like a gadget than a feature.

Anyway. Enabling Telegram support needs at least uncomment of define USE_TELEGRAM and uncomment of either define USE_TELEGRAM_FINGERPRINT OR define USE_MQTT_TLS_CA_CERT

Default, once enabled in user_config_override it does nothing. It needs to know your bots token and chat id. Enter them with command TmToken and TmChatId.

There are three state commands allowing control over the comms.

Default polling time is 10 seconds but can be changed with command TmPoll from 4 to 300 seconds.

Complete command syntax can be found in the header of file xdrv_40_telegram.ino

EDIT: Command TmState options are NOT persistent so if needed use rules to keep them in a state you like.

s-hadinger commented 4 years ago

Nice !!!

ademirci commented 4 years ago

I really admire you guys. Thanks for this great tasmota software and thanks for this great effort to enable tasmota to send and receive telegram messages. I will test this as soon as possible and share my experience.

ademirci commented 4 years ago

I just test this and working well. I can send commands to tasmota and receive messages from tasmota to my telegram account at my phone. Thanks again. I close this request.

JackNJack commented 4 years ago

I experimented this almost a 3weeks now. I observed that telegramClient->connect("api.telegram.org", 443) is triggering exception 29 whenever my connection to internet is slow. Also, it takes almost 2sec (sometimes 30sec even if I included 1sec timeout in my telegram request String command = "bot" + _token + "/getUpdates?offset=" + offset + "&limit=1&timeout=1";) to connect. Which is why some of the drivers like the webserver is also slow on its response. Can anyone show me a way to solve this? Or is it possible to multithread the process so that I will only process the request result if it is already available and entertain other drivers while waiting for the result.

s-hadinger commented 4 years ago

30s is very long. Do you know if the time is spent at the TLS connection or later at HTTP level?

You could remove comments on line 105, to have a precise timing of TLS connection, and even move to LOG_LEVEL_INFO:

    AddLog_P2(LOG_LEVEL_DEBUG, PSTR("TGM: Connected in %d ms, max ThunkStack used %d"), millis() - tls_connect_time, telegramClient->getMaxThunkStackUse());
JackNJack commented 4 years ago

AddLog_P2(LOG_LEVEL_DEBUG, PSTR("TGM: Connected in %d ms, max ThunkStack used %d"), millis() - tls_connect_time, telegramClient->getMaxThunkStackUse());

Thanks for the reply @s-hadinger, I actually uncommented that part and it shows ~2000ms, sometimes 30000ms. 18:38:46 TGM: Connected in 21335 ms, max ThunkStack used 4516 TGM: Connected in 1941 ms, max ThunkStack used 4516 --> This is my usual connection about 1800 to 1900+

I also put a log before telegramClient->connect("api.telegram.org", 443) and it is the last log shown whenever an exception 29 triggers.

s-hadinger commented 4 years ago

So it is definitely a problem during TLS connection. 21s is far too long.

You can try to temporarily double buffers, I suspect this may have some side effects (warning it eats up more RAM):

  static const uint32_t tls_rx_size = 4096;
  static const uint32_t tls_tx_size = 2048;
JackNJack commented 4 years ago

So it is definitely a problem during TLS connection. 21s is far too long.

You can try to temporarily double buffers, I suspect this may have some side effects (warning it eats up more RAM):

  static const uint32_t tls_rx_size = 4096;
  static const uint32_t tls_tx_size = 2048;

Thanks @s-hadinger , i'll try it. Based on Info: My RAM has 20KB free. When i dont use telegram it would go up to 28KB.

JackNJack commented 4 years ago

@s-hadinger my RAM went down to 15KB free, but the same performance. Response in 2sec nominal, while sometimes it reached 5sec to 15sec TGM: Connected in 14571 ms, max ThunkStack used 4484

s-hadinger commented 4 years ago

Ok, it's not the problem then. Logs for bearssl are quite obscure, I don't have any easy solution.

Just to make sure, is the ESP running at 160MHz instead of standard 80 MHz?

arendst commented 4 years ago

Just to note mine runs fine on 80MHz with quick responses around 2 Secs.

JackNJack commented 4 years ago

Mine also runs at 80Mhz. When I reduce both TLS rx/tx size to 512 it performs much better with 1800ms to 2000ms response. But when internet is slow it goes up to 5sec. I reduce the size to 512 since in my application I only receive up to 10 chars of text with single message each GET.

s-hadinger commented 4 years ago

Good to know that lowering the buffer size actually improves performance. I would still try 160 Mhz.

riker65 commented 4 years ago

Hi, great news with the telegram support but where do I find docu how to configure

thanks for guiding

ascillato commented 4 years ago

where do I find docu how to configure

https://github.com/arendst/Tasmota/issues/8619#issuecomment-642764152