Closed dizcza closed 1 month ago
dizcza
commented
2 months ago Fixed by increasing the keep-alive timeout on the server side.
In particular, I added --keep-alive 10 to my gunicorn command.
Not sure if I need to enable the keep-alive in the ESP HTTP client config -- added it to be safe.
dizcza
commented
2 months ago Increasing keep alive helps but hasn't fully resolved the problem. Sometimes I still get
E (15:56:57.241) esp-x509-crt-bundle: PK verify failed with error FFFFBD70
E (15:56:57.245) esp-x509-crt-bundle: Failed to verify certificate
E (15:56:57.255) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x3000
E (15:56:57.272) HTTP_CLIENT: Connection failed
W (15:56:57.275) app_http: app_http_client_get_tasks status = 0, content_length = 0or
E (19:27:53.717) esp-x509-crt-bundle: PK verify failed with error FFFFBD70
E (19:27:53.722) esp-x509-crt-bundle: Failed to verify certificate
E (19:27:53.729) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x3000
E (19:27:53.732) esp-tls: Failed to open new connection
E (19:27:53.737) transport_base: Failed to open a new connection
E (19:27:53.754) HTTP_CLIENT: Connection failed, sock < 0
W (19:27:53.757) app_http: app_http_client_get_tasks line 500: ESP_ERR_HTTP_CONNECT
nileshkale123
commented
2 months ago dizcza
commented
1 month ago @nileshkale123 I'm not using the https_request API. I'm using esp_http_client API for both GET and POST requests.
I guess this corresponds to setting cert_pem:
void ota_http_client_config_init(esp_http_client_config_t *config) {
extern const char ca_cert_pem_start[] asm("_binary_ca_cert_pem_start");
extern const char ca_cert_pem_end[] asm("_binary_ca_cert_pem_end");
memset(config, 0, sizeof(esp_http_client_config_t));
config->username = ENV_OTA_SERVER_USER;
config->password = ENV_OTA_SERVER_PASS;
config->auth_type = HTTP_AUTH_TYPE_BASIC;
config->timeout_ms = OTA_LOADER_TIMEOUT_MS;
config->skip_cert_common_name_check = true;
config->cert_pem = ca_cert_pem_start;
config->cert_len = ca_cert_pem_end - ca_cert_pem_start;
config->transport_type = HTTP_TRANSPORT_OVER_SSL;
}As you might have guessed, I'm using the same client config for both HTTPS OTA updates and HTTPS GET/POST client requests.
And I still get the same errors
W (14:33:10.300) app_http: Last esp error code: ESP_ERR_MBEDTLS_SSL_SETUP_FAILED
W (14:33:10.303) app_http: Last mbedtls failure: 0x7f00
E (14:33:20.250) esp-tls-mbedtls: mbedtls_ssl_setup returned -0x7F00
E (14:33:20.255) esp-tls: create_ssl_handle failed
E (14:33:20.259) esp-tls: Failed to open new connection
E (14:33:20.262) transport_base: Failed to open a new connection
E (14:33:20.272) HTTP_CLIENT: Connection failed, sock < 0And on the server side
Traceback (most recent call last):
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 65, in handle
util.reraise(*sys.exc_info())
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/util.py", line 626, in reraise
raise value
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 48, in handle
req = next(parser)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/parser.py", line 42, in __next__
self.mesg = self.mesg_class(self.cfg, self.unreader, self.source_addr, self.req_count)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/message.py", line 180, in __init__
super().__init__(cfg, unreader, peer_addr)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/message.py", line 54, in __init__
unused = self.parse(self.unreader)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/message.py", line 192, in parse
self.get_data(unreader, buf, stop=True)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/message.py", line 183, in get_data
data = unreader.read()
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/unreader.py", line 37, in read
d = self.chunk()
File "/home/vps/miniconda3/lib/python3.7/site-packages/gunicorn/http/unreader.py", line 64, in chunk
return self.sock.recv(self.mxchunk)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gevent/_ssl3.py", line 555, in recv
return self.read(buflen)
File "/home/vps/miniconda3/lib/python3.7/site-packages/gevent/_ssl3.py", line 386, in read
return self._sslobj.read(nbytes or 1024)
OSError: [Errno 0] ErrorHello @dizcza
The error code 0x7F00 relates to a heap memory issue, indicating that there might not be enough memory available during the SSL setup phase. Could you please check the free heap memory before initiating the connection, especially just before the esp_http_client configuration is set up?
If the available memory is low, you might consider optimizing memory usage or increasing the heap size, if possible.
Additionally, could you please share a small executable example using an IDF project so that we can reproduce the issue and provide further assistance?
dizcza
commented
1 month ago Hmm. It seems I cannot reliably reproduce the issue: it happened only once or twice daily during the last two days.
Could it be due to changing esp_crt_bundle_attach to cert_pem... hard to tell.
I need to add another ESP32 device to talk to the same server and see what happens.
I'll reopen the issue if I get more description.
dizcza
commented
3 weeks ago Ok, after I
esp_crt_bundle_attach to cert_pemthe problem disappeared.
Either of the steps alone didn't solve the problem. Only when I did all things, the issue was gone. Make sure you're not running low on the internal heap memory:
heap_caps_get_minimum_free_size(MALLOC_CAP_8BIT | MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL).
Keep track of this value for hours.
Answers checklist.
IDF version.
v5.1.4-700-ge026fd1f81
Espressif SoC revision.
ESP32
Operating System used.
Linux
How did you build your project?
Command line with idf.py
If you are using Windows, please specify command line type.
None
Development Kit.
TTGO T8
Power Supply used.
USB
What is the expected behavior?
esp_http_client_performresults in ESP_OK with no mbedtls errorsWhat is the actual behavior?
Steps to reproduce.
Debug Logs.
More Information.
When I put the code of
app_http_client_loopto theesp_http_clientexample code (in thehttps_with_url()function), I don't get these errors or at least I wasn't able to reproduce them for several minutes. Neither do I get them when I don't use SSL stack (it works over HTTP but not HTTPS).Other times I get this repeating error:
I get these errors even if I strip 80% of the functionality of my app. I don't understand where and why the issue happens.
mbedtls: ssl_tls.c:1420 alloc(16717 bytes) failedpoints to some memory issues but the first logs I sent don't have alloc issues at all!Here is the modified example and the logs:
In both cases, I'm using the same
sdkconfig.defaults(see https://github.com/espressif/esp-idf/issues/14467#issuecomment-2320515015) and the same HTTPS endpoint.This code works fine with no errors: