Save heap memory by opening two socket connections using the same certificate?

[ranr]

Hi,

I am trying to use two HTTPS connections simultaneously on the ESP8266, both of them with persistent HTTPS connection (so the sockets stay connected). I saw that other people also mentioned the large heap memory footprint and ran into that problem myself too. Before the first socket is opened, I have 31,648 bytes and it drops to 20,872. The second socket connection drops the free heap space further to 9792 bytes. So about 11K per socket connection.

I was wondering what is the reason for this relatively big footprint? Is it possible to skip the verify() of the fingerprint for one of the servers (for which I don't need authentication) and save memory? Another option I am considering - if the main memory allocation is due to the certificate verification, is it possible to have both sockets use the same certificate verification, so that I would be allocating only 11K instead of 22K?

One thing I've noticed is that the call to ssl_ctx_new() is performed only once, and that the massive memory allocation happens on the two calls to ssl_client_new() - about 8Kb.

Thanks

[igrr]

axTLS heap usage consists of the following components:

• TLS fragment buffer. Default size is about 6kB, it is grown dynamically up to the maximum TLS fragment size of 16kB, if the server doesn't support fragment length negotiation extension (RFC 6066).
• Certificates received from the server. As an optimization, these are purged as soon as TLS fragment buffer needs to be resized (in HTTPS scenario, this usually happens when the server starts sending HTTP response). Size: depends on the server's certificate chain.
• CA certificates, if provided by the client. Size: depends on CA certificate, usually about 1-2kB.
• Bigint context for RSA. This is kept around throughout the connection. Size: not sure, but can be determined by instrumenting bigint functions.
• The rest of TLS session state: keys, nonces, etc.
• Large amount of bigint values needed during key exchange and certificate verification. These are freed after the handshake, but may cause an out-of-memory condition during the handshake. Size: depends on the RSA key length.

Is it possible to skip the verify() of the fingerprint for one of the servers (for which I don't need authentication) and save memory?

As far as i understand, axTLS still loads the whole certificate chain into memory, as it is not aware of the method which will later be used for certificate verification. This is done in process_certificate (tls1.c).

is it possible to have both sockets use the same certificate verification, so that I would be allocating only 11K instead of 22K?

Each concurrent TLS connection needs to do handshake and validate certificates separately. However, if a session has already been established, new connection can perform abbreviated handshake. Unfortunately, axTLS doesn't have support for this.

To summarize, without significant changes to axTLS, you can get biggest memory saving effect if you enable fragment length negotiation extension on the server side.

[slaff]

Each concurrent TLS connection needs to do handshake and validate certificates separately. However, if a session has already been established, new connection can perform abbreviated handshake. Unfortunately, axTLS doesn't have support for this.

Actually axTLS supports abbreviated handshake or SSL resumption as is the official term. What you need to do is keep the SSL session id from a previous session and then use it in a new SSL to the same server providing the SSL session id in the SSL context. We have this implemented in Sming and it is working like charm.

There is even better way to decrease the SSL overhead - inform the server to keep the TCP connection open and then make another request using the same SSL. If you are interested how this can be implemented take a look at the HttpClient in Sming.

[igrr]

Thanks @slaff, looks really useful! Session ID support should be added to Arduino wrapper as well.

I think the equivalent of HttpClient in Arduino does support TCP connection reuse, but the OP was asking about having two connections in parallel (not sure whether they use same server or not).