Adding `esp-aws-iot` as submodule (CA-151)

[C47D]

Hi,

We're trying to add this repository (and all it's submodules) to our proyect, we're interested in the OTA MQTT example code, so the ota-beta/202103.00 is being tracked, the master branch of aws-iot-device-sdk-embedded-C is also being tracked.

After setting those branches up, I've applied the patch available in components\aws\examples\ota\ota_mqtt\patches like so:

components\esp-aws-iot\aws-iot-device-sdk-embedded-C> git apply ..\examples\ota\ota_mqtt\patches\aws-iot-device-sdk-embedded-C.patch

And copied the missing header file into my esp-idf installation:

cp port/crypto/threading_alt.h ~/esp/esp-idf/components/mbedtls/port/include/

Then enabled the FreeRTOS Static API and threading abstraction layer (Enable threading alternate implementation) of mbedTLS.

I think we're close to being able to compile the project but we're missing some files:

ccache: error: Failed to create temporary file for esp-idf/aws/CMakeFiles/__idf_aws.dir/aws-iot-device-sdk-embedded-C/libraries/aws/ota-for-aws-iot-embedded-sdk/source/dependency/3rdparty/tinycbor/src/cborencoder_close_container_checked.c.obj: No such file or directory

...

../components/aws/port/crypto/iot_crypto.c:52:30: error: implicit declaration of function 'configPRINTF'; did you mean 'ETS_PRINTF'? [-Werror=implicit-function-declaration]

Find the full output log attached.

do you think we're missing some step in the project setup?

log_failed_compilation.txt

[C47D]

UPDATE

I can now compile the project, using some tricks, please let me know if you find something suspicious:

In components/aws/port/crypto/iot_crypto.c I have the following diff, configPRINTF should be defined in FreeRTOSConfig.h and it is not:

diff --git a/port/crypto/iot_crypto.c b/port/crypto/iot_crypto.c
index b992cfb..45d9e47 100644
--- a/port/crypto/iot_crypto.c
+++ b/port/crypto/iot_crypto.c
@@ -49,7 +49,8 @@

-#define CRYPTO_PRINT( X )    configPRINTF( X )
+#define CRYPTO_PRINT( X )

Then copied the Kconfig.projbuild file from esp-aws-iot/examples/ota/ota_mqtt/main into my project main directory.

Last thing I had to so is compile the application using idf.py --no-ccache build, the application doesn't compile with ccache enabled.

[C47D]

UPDATE

After several tries we went and tracked the release/beta branch of this repo, we noticed the missing CMakeLists.txt file in the root directory so we created one.

include(${PROJECT_DIR}/components/esp_aws_iot/port/portFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/aws/jobs-for-aws-iot-embedded-sdk/jobsFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/aws/ota-for-aws-iot-embedded-sdk/otaFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/aws/device-shadow-for-aws-iot-embedded-sdk/shadowFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/standard/corePKCS11/pkcsFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/standard/coreMQTT/mqttFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/standard/coreJSON/jsonFilePaths.cmake)
include(${PROJECT_DIR}/components/esp_aws_iot/aws-iot-device-sdk-embedded-C/libraries/standard/backoffAlgorithm/backoffAlgorithmFilePaths.cmake)

# Set path to corePKCS11 and it's third party libraries.
set(COREPKCS11_LOCATION "aws-iot-device-sdk-embedded-C/libraries/standard/corePKCS11")
set(3RDPARTY_LOCATION "aws-iot-device-sdk-embedded-C/libraries/source/dependency/3rdparty")
set(CORE_PKCS11_3RDPARTY_LOCATION "${COREPKCS11_LOCATION}/source/dependency/3rdparty")

set(COMPONENT_SRCS
    "${PORT_SOURCES}"
    "${PORT_OTA_SOURCES}"
    "${JOBS_SOURCES}"
    "${OTA_SOURCES}"
    "${OTA_OS_FREERTOS_SOURCES}"
    "${OTA_MQTT_SOURCES}"
    "${SHADOW_SOURCES}"
    "${BACKOFF_ALGORITHM_SOURCES}"
    "${JSON_SOURCES}"
    "${MQTT_SOURCES}"
    "${MQTT_SERIALIZER_SOURCES}"
    "${PKCS_SOURCES}"
    "${CORE_PKCS11_3RDPARTY_LOCATION}/mbedtls_utils/mbedtls_utils.c"
    "${CORE_PKCS11_3RDPARTY_LOCATION}/mbedtls_utils/mbedtls_error.c"
    "aws-iot-device-sdk-embedded-C/platform/posix/clock_posix.c"
    )

set(COMPONENT_ADD_INCLUDEDIRS
    "."
    "${PORT_INCLUDE_PUBLIC_DIRS}"
    "${PORT_OTA_INCLUDE_PUBLIC_DIRS}"
    "${JOBS_INCLUDE_PUBLIC_DIRS}"
    "${OTA_INCLUDE_PUBLIC_DIRS}"
    "${OTA_INCLUDE_PRIVATE_DIRS}"
    "${OTA_INCLUDE_OS_FREERTOS_DIRS}"
    "${SHADOW_INCLUDE_PUBLIC_DIRS}"
    "${BACKOFF_ALGORITHM_INCLUDE_PUBLIC_DIRS}"
    "${JSON_INCLUDE_PUBLIC_DIRS}"
    "${MQTT_INCLUDE_PUBLIC_DIRS}"
    "${PKCS_INCLUDE_PUBLIC_DIRS}"
    "${CORE_PKCS11_3RDPARTY_LOCATION}/pkcs11"
    "${CORE_PKCS11_3RDPARTY_LOCATION}/mbedtls_utils"
    "aws-iot-device-sdk-embedded-C/platform/include/"
    )

idf_component_register(SRCS "${COMPONENT_SRCS}"
    INCLUDE_DIRS ${COMPONENT_ADD_INCLUDEDIRS}
    REQUIRES lwip nghttp mbedtls tcp_transport spi_flash bootloader_support app_update main
    )

target_compile_definitions(${COMPONENT_LIB} PUBLIC "-DSHADOW_DO_NOT_USE_CUSTOM_CONFIG")

# If we uncomment the following line we got a bunch of messages about redefined symbols
#target_compile_definitions(${COMPONENT_LIB} PUBLIC "-DOTA_DO_NOT_USE_CUSTOM_CONFIG")

We can build the project using the following command idf.py --no-ccache build, without the --no-ccache flag we can't because the ota cbor source files are not being found, I'm still investigating why.

My question is, is the user expected to provide their CMakeLists.txt file inside this repo or is it planned to be added later?

Regards

[SolidStateLEDLighting]

C47D, thank you very much for your efforts here. I think a lot of us are looking for Espressif to get this new version of the software up and running.