change(esp-wolfssl): allow to enable OCSP support

frankencode commented 5 months ago

I've added an option (default off) to allow to enable OCSP with esp-wolfssl. In particular it makes wolfSSL process the OCSP status forwarded in stabling v1 format during TLS handshake.

CLAassistant commented 5 months ago

All committers have signed the CLA.

gojimmypi commented 5 months ago

Hi @frankencode and thank you for your interest in wolfssl.

Please note that this espressif/esp-wolfssl is stale and not managed by wolfssl as mentioned in https://github.com/espressif/esp-idf/pull/13618#issuecomment-2058148457.

We'd like to invite you to instead open pull requests or issues at wolfSSL/wolfssl.

Additionally, please note that we have wolfSSL available as a Managed Component as well as some examples using a GitHub clone of wolfssl as a component in the ESP-IDF with only a CMakeLists.txt.

frankencode commented 5 months ago

@gojimmypi

Hi gojimmypi!

I know. I just watched your video on setting up wolfssl on esp yesterday. It didn't help much though. I need to have wolfSSL work as a proper esp-idf component. When I use your esp-wolfssl from espressif component store it won't allow me to select wolfSSL as an esp-tls backend in idf.py menuconfig. The reason is among other things a missing Kconfig file and by the looks general lack of support for esp-tls. Therefore I actually using this stale repository here, because it actually works out of the box and allows me to use wolfSSL to drive espressif's communication stack. For instance eps-websockets <-> eps-tls (using esp_tls_wolfssl) <-> wolfssl. There are also other problems with the other two solution you point out. The biggest one is that the configurations that you ship currently lead to a way to big wolfSSL library. When you enable WiFi, BLE, websockets, etc. and wolfSSL, the way wolfSSL ships currently by default, it won't fit anymore on a standard 1MB application partition (and in general is the biggest part of the system). I also managed to get wolfssl 5.7 from github run with the user_config.h from this repository. I'm currently only doing explorative prototyping... The reason I post a few line changes here is to communicate to espressif further changes which might be needed in the esp_tls_wolfssl wrapper.

God speed, Frank

gojimmypi commented 5 months ago

@frankencode

Hi Frank!

I need to have wolfSSL work as a proper esp-idf component.

Ah yes, have you tried copying the wolfssl component directory to the ESP-IDF directory? For me using VisualGDB on Windows this would be: C:\SysGCC\esp32\esp-idf\v5.2\components. I've not recently tested this, but I would expect it to work.

In the components directory, the respective CMakeLists.txt will need a setting as to where the wolfSSL source code is located, or the WOLFSSL_ROOT environment variable can be set.

This might be the best solution if you are using the ESP-IDF. The Managed Component will of course not be appropriate as the user_settings.h file needs to be editable.

The reason is among other things a missing Kconfig file and by the looks general lack of support for esp-tls

You are correct and I need to give more attention to this.

In the meantime, is it possible for you to use the TLS capabilities from wolfSSL directly and avoid the esp-tls layer?

it won't fit anymore on a standard 1MB .. that the configurations that you ship currently lead to a way to big wolfSSL library

The wolfSSL libraries can certainly be configured for small memory footprints. Recently I was able to get wolfSSL TLS working on the Arduino Nano 33 IoT SAMD21 in only 32KB RAM + 256KB flash. See X/Tweet micro-blog.

There's also an example Arduino user_settings.h over in examples/configs.

That said, you are also correct about the default user_settings.h turning on too many features if a small flash footprint is desired. Perhaps I should include several user settings example files, each with different objectives.

currently only doing explorative prototyping

Cool. I'd like to help.

On a somewhat related note, I've been working on wolfSSL for Arduino as well as wolfSSL for PlatformIO (currently only the Arduino version) and staging wolfSSL for PlatformIO with the latest code to help with rapid prototyping for many different environments and platorms.

Also, if you are interested in Post Quantum, I have an Open PR https://github.com/wolfSSL/wolfssl/pull/7392 that updates the Espressif Examples, including adding support for Kyber KEM.

Can you share more details of your project? Feel free to contact us at support@wolfssl.com

In the meantime I'll take a look at the esp-tls. If you have some sample code with your prototype objective, that would be great.

Cheers

frankencode commented 5 months ago

Ah yes, have you tried copying the wolfssl component directory to the ESP-IDF directory? For me using VisualGDB on Windows this would be: C:\SysGCC\esp32\esp-idf\v5.2\components. I've not recently tested this, but I would expect it to work.

YES. Tried that copy script. Like I said it is missing the Kconfig providing the TLS_STACK_WOLFSSL variable. Without I cannot select wolfSSL in the idf.py menuconfig under menu point "components => esp-tls".

In the components directory, the respective CMakeLists.txt will need a setting as to where the wolfSSL source code is located, or the WOLFSSL_ROOT environment variable can be set.

That's unneeded I think. I'm not sure why doing it different from any other esp-idf component.

This might be the best solution if you are using the ESP-IDF. The Managed Component will of course not be appropriate as the user_settings.h file needs to be editable.

Yes. I used the user_settings from this repository and updated it to reflect changes up to 5.7.

The reason is among other things a missing Kconfig file and by the looks general lack of support for esp-tls

You are correct and I need to give more attention to this.

(There are a lot of platforms you have to keep track of;-)

In the meantime, is it possible for you to use the TLS capabilities from wolfSSL directly and avoid the esp-tls layer?

ABSOLUTELY NOT. Espressif's network protocol implementations are build on top of esp-tls. For instance have a look at the websocket client example of https://components.espressif.com/components/espressif/esp_websocket_client . I works out of the box with wolfSSL from this repository. (Kconfig problem as I mentioned earlier.)

The wolfSSL libraries can certainly be configured for small memory footprints. Recently I was able to get wolfSSL TLS working on the Arduino Nano 33 IoT SAMD21 in only 32KB RAM + 256KB flash. See X/Tweet micro-blog.

Yes, I will/have to trim it. My customer wants to use it primarily because of OCSP support (and because they are already happy customer at wolfSSL;-)

In the meantime I'll take a look at the esp-tls. If you have some sample code with your prototype objective, that would be great.

Like I said I was just testing the esp_websocket_client library. I would be cool if its client example would work out of the box with the wolfSSL from components.espressif.com . I also posted some fixes on the espressif side to better support wolfSSL, have a look: https://github.com/espressif/esp-idf/pull/13618

gojimmypi commented 5 months ago

Hi @frankencode

Thank you so much for the details.

it is missing the Kconfig providing the TLS_STACK_WOLFSSL variable

I'll definitely get that fixed right away.

In the components directory, the respective CMakeLists.txt will need a setting as to where the wolfSSL source code is located, >> or the WOLFSSL_ROOT environment variable can be set.

That's unneeded I think. I'm not sure why doing it different from any other esp-idf component.

The primary reason this is different than other ESP-IDF components is licensing and upstream owners. In particular, I designed the CMakeLists.txt to be able to point at wolfSSL source so that the actual wolfSSL code being used is in a wolfssl clone of a fork. (and not installed as a copy in a local project, as a copy in ESP-IDF, etc.)

This way, when changes are made to the wolfSSL code, it is already in place and ready to commit and create a PR.

The current examples in the wolfSSL recurse up the directory tree looking for wolfSSL source code.

To really appreciate this, you should have seen the convoluted process that I had which copied code from the local install back and forth with the actual repository code. That became quite unreasonable, at least for me.

That's unneeded I think.

For most users that do not intend on editing the wolfSSL source code, you are probably correct.

Perhaps there's a benefit of being able to easily change source versions... or ensure that all projects are using exactly the same version.

Espressif's network protocol implementations are build on top of esp-tls. For instance have a look at the websocket client example.

Ah yes, good point if you need to use the web sockets rather than the lower-level lwip/sockets, see how the wolfSSL component needs to be seen by ESP-IDF.

I'm not sure if wolfSSL has a web sockets layer. Let me check into that and get back to you.

they are already happy customer at wolfSSL

A name you can share? Feel free to contact me directly: jim (at) wolfssl.com, or support@wolfssl.com

Depending on the customer and respective support level, we might be able to dedicate more resources to help you.

In the meantime, I'll be taking a closer look at the esp-tls, Kconfig files, any your other suggestions which are all completely valid and I would definitely like to have working. Thanks again.

Cheers

gojimmypi commented 4 months ago

@frankencode

Hi Frank -

First, regarding websockets, check out https://github.com/wolfssl/osp, in particular wolfSSL/osp/websocketpp.

I do not personally have any experience with that, but let me know if you are interested in using something other than the esp-tls layer in your project.

Regarding the esp-tls....

My example wolfssl_client has been updated with a Kconfig that has the esp-tls TLS_STACK_WOLFSSL capability with wolfSSL in the Espressif components and not a local component.

I would have used your ESP-IDF fork at frankencode/esp-idf but I got into quite a tangle with the ESP-IDF v5.2 vs the v5.3 in master.

I do have the esp-13.2.0_20240305 but it is not found looking in esp-13.2.0_20230928:

-- Compiler supported targets: xtensa-esp-elf
CMake Error at C:/SysGCC/esp32/esp-idf/v5.2-gojimmypi/tools/cmake/tool_version_check.cmake:36 (message):

  Tool doesn't match supported version from list ['esp-13.2.0_20240305']:
  C:/SysGCC/esp32/tools/xtensa-esp-elf/esp-13.2.0_20230928/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc.exe

As such, for now I've made some in-place edits in my ESP-IDF v5.2 and commented below.

Here are the steps to use:

1. Rename the local project `components` directory to `components.bak`

This will disable wolfSSL as a local project component.

2. Copy the wolfssl directory and contents to ESP-IDF components.

My ESP-IDF components are located in C:\SysGCC\esp32\esp-idf\v5.2\components

C:\SysGCC\esp32\esp-idf\v5.2\components\wolfssl>dir /s

04/17/2024  09:16 AM            36,129 CMakeLists.txt
04/15/2024  05:09 PM            11,320 component.mk
04/17/2024  11:01 AM    <DIR>          include
04/16/2024  02:43 PM             1,048 Kconfig

 Directory of C:\SysGCC\esp32\esp-idf\v5.2\components\wolfssl\include

04/01/2024  10:01 AM               859 config.h
04/17/2024  11:01 AM            20,940 user_settings.h

3. Edit the wolfssl CMakeLists.txt in the new directory

My file is in C:\SysGCC\esp32\esp-idf\v5.2\components\wolfssl

On or about line 37 should be a comment to manually set the source for wolfSSL.

My wolfSSL source is my clone located in C:\workspace\wolfssl-gojimmypi-pr

# Optionally set your source to wolfSSL in your project CMakeLists.txt like this:
set(WOLFSSL_ROOT "C:/workspace/wolfssl-gojimmypi-pr" )

4. Edit esp-tls component CMakeLists.txt

Mine is in C:\SysGCC\esp32\esp-idf\v5.2\components\esp-tls

On or about line 26 should be a if(CONFIG_ESP_TLS_USING_WOLFSSL).

Replace the esp-wolfssl with wolfssl. The old line with esp-wolfssl is commented out:

if(CONFIG_ESP_TLS_USING_WOLFSSL)
#   idf_component_get_property(wolfssl esp-wolfssl COMPONENT_LIB)
    idf_component_get_property(wolfssl wolfssl COMPONENT_LIB)
    target_link_libraries(${COMPONENT_LIB} PUBLIC ${wolfssl})
endif()

5. Edit esp_tls_wolfssl.c

Mine is in C:\SysGCC\esp32\esp-idf\v5.2\components\esp-tls

Add these lines at the beginning of the file. I added mine after the netdb include.

#include <wolfssl/wolfcrypt/settings.h>
#include <wolfssl/ssl.h>
#define OPENSSL_EXTRA
#include <wolfssl/openssl/x509.h>

6. Edit esp_tls_crypto.c

Mine is in C:\SysGCC\esp32\esp-idf\v5.2\components\esp-tls\esp-tls-crypto

Include the wolfssl includes after the #elif CONFIG_ESP_TLS_USING_WOLFSSL

#ifdef CONFIG_ESP_TLS_USING_MBEDTLS
    #include "mbedtls/sha1.h"
    #include "mbedtls/base64.h"
    #define _esp_crypto_sha1 esp_crypto_sha1_mbedtls
    #define _esp_crypto_base64_encode esp_crypto_bas64_encode_mbedtls
#elif CONFIG_ESP_TLS_USING_WOLFSSL
    #define OPENSSL_EXTRA
    #include "wolfssl/wolfcrypt/settings.h"
    #include "wolfssl/ssl.h" /* SHA functions are listed in wolfssl/ssl.h */
    #include "wolfssl/openssl/sha.h" /* old SHA functions only available with OpenSSL */
    #include "wolfssl/wolfcrypt/coding.h"
    #define _esp_crypto_sha1 esp_crypto_sha1_wolfSSL
    #define _esp_crypto_base64_encode esp_crypto_base64_encode_woflSSL
#endif

7. Special notes

It appears that the wolfSSL_SHA1 used in components/esp-tls/esp-tls-crypto/esp_tls_crypto.c:47:26 has been deprecated and is now only defined when OpenSSL compatibility is enabled. (e.g. #define OPENSSL_EXTRA)

The #include <wolfssl/wolfcrypt/settings.h> should be included everywhere wolfSSL is used and before any other wolfSSL include. Never include user_settings.h directly.

It appears there may be some oddity with the wolfSSL settings.h file when referenced from some of the ESP-IDF components as you may have noticed that I have an explicit #define for OPENSSL_EXTRA.

This is not a websockets demo yet, just something that compiles with the Kconfig file to enable wolfSSL in the esp-tls layer.

Please let me know if this works as desired.

Cheers

AdityaHPatwardhan commented 3 months ago

@frankencode Thanks for making the changes, just one small comment other wise the changes look good to me.

AdityaHPatwardhan commented 3 months ago

@frankencode Can you please merge the last two commits into one?

frankencode commented 3 months ago

@AdityaHPatwardhan: Sure, no problem. See updated head.

gojimmypi commented 3 months ago

Hi @frankencode - there will be an increased effort at improving the wolfSSL integration with the Espressif ESP-IDF.

I've opened this anchor issue https://github.com/espressif/esp-idf/issues/13966 to track progress.

espressif / esp-wolfssl