WiFiClientSecure is slow from release v2.5.0

[Teddyz]

Settings in IDE

• Module: Lolin/Wemos D1 mini
• Flash Size: 4MB
• lwip Variant: [v1.4|v2 Lower Memory|Higher Bandwidth]
• CPU Frequency: 80Mhz

Problem Description

Sending data via WiFiClientSecure has been very slow since v2.5.0. I use it with 256dpi/arduino-mqtt I have tested all lwip-settings and they make only little difference.

Sending 4 mqtt-messages takes: WiFiClientSecure + v2.4.2 = 5ms WiFiClient + v2.6.3 = 2ms WiFiClientSecure + v2.6.3 = 780ms

Attached is the modified example code from the mqtt library that I tested with. Uncomment net.setInsecure(); if using with newer core than v2.4.2. AdafruitHuzzahESP8266Secure_DT20200202.zip

Thanks for looking at this problem.

[Teddyz]

When testing today I found that the difference seems to be after sending the encrypted TCP packet to the MQTT-broker, something is now waiting until it receives the ACK. In v2.4.2 it immediately released control back to the library. The 200ms delay seems to a large degree be from waiting for the ACK and not actually from sending the packet. I also wrote about it here: https://github.com/256dpi/arduino-mqtt/issues/165#issuecomment-583590337 It would be great if you could find a way to let us who are not interested in the ACK avoid this waiting.

[earlephilhower]

Sorry, but we can't adjust how BearSSL handles the SSL protocol. If you want to, you can use the axtls class instead, which may not wait for this ACK. See the examples for how to call the older, obsolete version.

That said, though, TCP ACKs are below the SSL protocol itself so I think you might be looking at a LWIP issue. @d-a-v might be able to assist here.

[Teddyz]

Thanks for looking at my problem! I managed to do the needed changes in WiFiClientSecure.h and now it seems to be working the old way. For me it is "good enough". RAM usage is lower, so heap fragmentation went from 28% to 8%, heap free from 9k to 25k and heap maxblocksize from 5k to 23k. Establish connection to api.pushover.net is not much faster, but sending the data and receiving OK is faster (from 1200ms to 600ms). But best of all for me is that ssl communication no longer interferes with accelstepper library.

I have a work-around, so OK to close from my point of view.

[devyte]

Closing then. Out of curiosity, what did you change in WiFiClientSecure.h? The references you point to describe a change in the Windows registry, but not changes to the header.

[Teddyz]

I didn't want to study this, but here we are. TCP/IP communication has a mechanism that replies with an ACK for every packet it has received. Many packets equals many ACKs, so it collects a few of them in one packet to send back to minimize this type of traffic. A Windows computer, I think, wants to have at least two ACKs before it sends them back. If only one is received, it patiently waits 200ms until it gives up and returns this lonely ACK.

bearSSL, however, seems to demand the ACK before even sending a second TCP packet. Combining these two ways of working, and we have got ourselves a limitation of 5 TCP packets per second. This is for SSL encrypted traffic, all other TCP traffic goes fast and well. (Caveat: this is my understanding. Since noone else seems to think this is unacceptable, I am sure the problem is at my side). Anyway, my first workaround was to use the registry-hack to tell my Windows-computer to send the ACK, even if there is only one to send. This helped with the servers I have control over. The better (for me) solution was to revert back to the obsolete AXTLS, meaning no more TLSv1.2. This was done by changing WiFiClientSecure.h to look like this.

#define USING_AXTLS
#include <ESP8266WiFi.h>
//#include <WiFiClientSecure.h>
#include "WiFiClientSecureAxTLS.h"
using namespace axTLS;

//#include "WiFiClientSecureBearSSL.h"

#ifndef USING_AXTLS
// do not default to BearSSL API ("using" has no "unusing" counterpart)
using namespace BearSSL;
#endif

Some reading that helped me understand the problem: https://www.extrahop.com/company/blog/2016/tcp-nodelay-nagle-quickack-best-practices/