search

espressif / esp-aws-iot

AWS IoT SDK for ESP32 based chipsets
Apache License 2.0
273 stars 164 forks source link

Beta ?? Can we get the CMakeList and KConfig files? (CA-165) #73

Open SolidStateLEDLighting opened 3 years ago

SolidStateLEDLighting commented 3 years ago

On Aug 27, your marketing department made a claim that all the IOT libraries were offered up for consumption.
https://www.espressif.com/en/news/LTSrelease?position=10&list=7PgGbNyNzkrX9xaSaTH_SK6ERYZtvTm9lBnXkNL03p0

The complete collection of libraries are in the beta release.

Don't we need to CMakeList and KConfig files to put that beta in use?

You're really not promoting the beta. Why? git clone --single-branch -b release/beta --recursive https://github.com/espressif/esp-aws-iot

There are certainly some weak and possibly unworkable things in the current master branch. I've been working in that for many months with mixed results.... and when I post about my problem with it -- I mostly get crickets.

Is the Beta working? If not -- then the company should post a retraction. If so, let's get it it out there and it will be trial by fire.

SolidStateLEDLighting commented 3 years ago

We are a commercial shop. We purchased about 8k ESP units this past year. Right now we use another 3rd party tool for software (which we intend to discontinue).

We will offer you feedback to assisting in verifying Shadow, MQTT, Https, Fleet Provisioning, and Jobs/OTA operations.

felixeriksson commented 3 years ago

I would also be very happy to have this, especially the OTA functionality. I've got a product built on esp-idf and pub/sub from this library, which is approaching release, and it needs to have OTA working. I'd be very happy to not have to invent this wheel again, especially if the OTA functionality in the beta is close to production ready.

SolidStateLEDLighting commented 3 years ago

I think the answers that we are all looking for are in the example builds -- we just expect too much of a Chinese company when it comes to support.

If you follow this press release page at the top of this post -- there is a set of instructions in there to build a thing_shadow project. Follow those instructions. That build has one error in it from an expired x509_cert_bundle. You'll need to go into menuconfig to turn off the "Enable trusted root certificate bundle". Follow the menu Component config->mbedTLS->Certificate Bundle->Enable trusted root certificate bundle.

That project will build. So, I expect that we could get these library to work if we follow the patterns in these example projects and instructions in the Readme.md files.

I have only just now built this one project and have done nothing else to make it run. I just wanted others to know that it looks like I'm headed in the right direction.

I suspect that the root directory won't need any config files because the intended configuration items are in those examples directories. I suspect we'll need to adopt the CMakeList.txt directives and then fish out the sdkconfig settings that apply to the example projects from those directories.

If my hunch is right - the examples directory is kind of what our "src" directory will be when the aws-iot-device-adk-embedded-C library is applied to a real project. If we adjust the paths inside the CMakeLists.txt, the I suspect we'll be in business.

It is entirely likely that the CMakeLists.txt files are a lot simpler due to the fact that we don't have a massive one at the root of this entire repository (as we have seen in the past and perhaps implicitly expect).

ameyinamdar commented 3 years ago

@SolidStateLEDLighting

  1. We have updated the CA certs in the ESP-IDF. Can you please check which tag/branch you're using and if you can pull the latest update in IDF that updates the CA certs bundle?
  2. Fully understood your intention to build AWS LTS libs as separate components. But if you see the structure of AWS provided libraries, they have a reverse dependency on application for the compilation. (for example, application needs to decide whether to use MQTT or HTTP for OTA and then library gets built accordingly) Hence, it's not quite possible to fit AWS libs into IDF component format. So we have provided application example that you can use as boilerplate structure for your own project. It should also be easy to keep the application outside of the AWS libraries.

We also will bring these libraries out of the "Beta" phase shortly. Going forward, these LTS libraries will be a primary way to use Espressif SoCs with AWS IoT and related services. Please feel free to let us know if you need something on priority and we can certainly consider.

HTH.

SolidStateLEDLighting commented 3 years ago

I just did a clone on this example 2 days ago -- when did you guys fix the CA certs bundle?

I have no particular intention to build as separate components. My desire is just to get the tool running in any way possible that fits into your intended build directives. Thank you for the extra information on the intended method of use. It looks like there is some "meat" in the Readme files and I have yet to examine those closely.

I think I would be plenty happy with just the primary functions. (client, shadow, fleet provisioning, jobs, OTA, pub, sub, MQTT, Https) Actually, I would say -- Please don't loose your focus on these core services until these are nailed down solidly. The ESP core virtues are these IOT functions. Without the wireless and IOT I would not be here.

And I sincerely hope you can keep the thread-safe multi-core supported in the wireless/IOT so we can really get a superior result. It is a real shame when you need to pin a process on a core and give up all the process sharing advantages. Just make sure the instruction that we need is available. Thanks.

dhavalgujar commented 3 years ago

I just did a clone on this example 2 days ago -- when did you guys fix the CA certs bundle?

Please find the relevant issue and the comment with more information on this here.

SolidStateLEDLighting commented 3 years ago

I just completely rebuilt the idf install (on Windows) and the certificate root bundle is still causing an error.

I renamed the esp-idf directory (which is on my desktop) I renamed the .espressif directory (which is on C://users//.espressif)

I do the complete install with the online installer -- it takes about 5 minutes and I have no complains during that install... everything runs nicely.

I return to my esp-aws-iot/examples/thing_shadow project and open it in VSC. It all works except for the failure with the cert root bundle....

certBundle
AdityaHPatwardhan commented 3 years ago

Hi @SolidStateLEDLighting, Can you please mention the commit id of esp-idf that you are currently at ? The issue was because of EC-ACC certificate which had negative serial number. Negative serial numbers are not supported by the latest version of cryptography. As @dhavalgujar already mentioned here The fix is already a part of esp-idf.

Just for confirming, can you please check if esp-idf/components/mbedtls/esp_crt_bundle/cacrt_all.pem file in your version of esp-idf has a certificate named EC-ACC. The certificate was removed with this commit.

Thanks

SolidStateLEDLighting commented 3 years ago

I don't work with commit ids so I don't know where to find this.

I just completely downloaded a fresh set of tools right before my last post here (as I described above). So, what commit id would this be?.... and obviously this fix could not be already part of the esp-idf that is downloaded by default.

YES -- that EC-ACC certificate is still inside the cacrt_all.pem file. My guess would be that somehow the certificate did not get removed.

AdityaHPatwardhan commented 3 years ago

Hi @SolidStateLEDLighting , I jest checked that the latest installer points to the v4.3.1 tag which is the latest release that we have. Unfortunately, the issue came after the release v4.3.1 and now the fix would be available only in release v4.3.2. The issue can be fixed if you change to any release/v4.x branches. But to get the fix in an official release, you might have to wait till v4.3.2.

1) One quick fix would be to remove the EC-ACC certificate from the file:esp-idf/components/mbedtls/esp_crt_bundle/cacrt_all.pem. Please refer this change. That should fix your issue.

Thanks for the issue, I hope this helps.

xeanhort commented 2 years ago

I'm facing the same error here. I applied this solution (4.3.1, Win10) and it seems to be fine regarding the certificate, but now it fails in other point. I'm not sure if is related or not, so I attach the full log here (I don't want to cut any important part): https://pastebin.com/zv48D5ye

Any help is appreciated, thank you! :)

SolidStateLEDLighting commented 2 years ago

"error: Failed to create temporary file for..."

The tools are in a strange mode and need to be reset.

  1. Do a "Reload Window"
  2. "Clean" and rebuild again.

From: Aaron @.> Sent: Wednesday, January 5, 2022 4:08 AM To: espressif/esp-aws-iot @.> Cc: keith ssledlighting.com @.>; Mention @.> Subject: Re: [espressif/esp-aws-iot] Beta ?? Can we get the CMakeList and KConfig files? (CA-165) (Issue #73)

I'm facing the same error here. I applied this solution (4.3.1, Win10) and it seems to be fine regarding the certificate, but now it fails in other point. I'm not sure if is related or not, so I attach the full log here (I don't want to cut any important part): https://pastebin.com/zv48D5ye

Any help is appreciated, thank you! :)

— Reply to this email directly, view it on GitHubhttps://github.com/espressif/esp-aws-iot/issues/73#issuecomment-1005134022, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGOKEY3UORFE7A2W6EWFQLUUNHVHANCNFSM5HPOZS7A. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.Message ID: @.***>

xeanhort commented 2 years ago

@SolidStateLEDLighting thank you for replying!

  1. I tried a "reload" (reboot Windows + Open IDF CMD again) but it seems to fail for the same reason: https://pastebin.com/RkcePrmg
  2. Also, I performed a "clean" and then a "build" but it seems that I have the same result: https://pastebin.com/jmk8Qfh9

I'm not sure if I need to reinstall the tools because if I move (in the same terminal) to one of my test projects, it builds successfully (https://pastebin.com/zk0jc77j) so maybe it's something related to this examples.

I'm building the OTA MQTT demo. Do you have to modify anything apart from certificates to make it work? Is there any other example from this repo that works out of the box?

Thank you!!

SolidStateLEDLighting commented 2 years ago

I was suggesting a reload of the window inside VS Code....

Here is my best suggestion for a Windows build environment. Follow ALL the default installation instructions when possible. Any time you deviate from the standard install, you risk having problems that you can't easily solve.

Use VS Code -- and start with the smallest example projects first --- like MQTT or Device Shadow (these do work right out of the box with the right certs).

I do NOT use any other tools (like PIO).... the default tool set is pretty decent and I'm very happy with it.

I believe that the OTA demo is the most difficult and I have not run it. I'm in the process of extracting the functionality from it because I feel that it is incompatible with the correct project structure for Fleet Provisioning and Device Shadow work (Jobs, etc).

Sorry I can't be of better help to you.

K.

From: Aaron @.> Sent: Wednesday, January 5, 2022 5:33 PM To: espressif/esp-aws-iot @.> Cc: keith ssledlighting.com @.>; Mention @.> Subject: Re: [espressif/esp-aws-iot] Beta ?? Can we get the CMakeList and KConfig files? (CA-165) (Issue #73)

@SolidStateLEDLightinghttps://github.com/SolidStateLEDLighting thank you for replying!

  1. I tried a "reload" (reboot Windows + Open IDF CMD again) but it seems to fail for the same reason: https://pastebin.com/RkcePrmg
  2. Also, I performed a "clean" and then a "build" but it seems that I have the same result: https://pastebin.com/jmk8Qfh9

I'm not sure if I need to reinstall the tools because if I move (in the same terminal) to one of my test projects, it builds successfully (https://pastebin.com/zk0jc77j) so maybe it's something related to this examples.

I'm building the OTA MQTT demo. Do you have to modify anything apart from certificates to make it work? Is there any other example from this repo that works out of the box?

Thank you!!

— Reply to this email directly, view it on GitHubhttps://github.com/espressif/esp-aws-iot/issues/73#issuecomment-1005522267, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGOKE43BS2TYEXVHD5GHBTUUQF7XANCNFSM5HPOZS7A. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.Message ID: @.***>

xeanhort commented 2 years ago

I was suggesting a reload of the window inside VS Code....

I'm using the "ESP-IDF 4.3 CMD" to open the console. I also use VSCode + ESP plugin, but I haven't used it with the demo projects.

Here is my best suggestion for a Windows build environment. Follow ALL the default installation instructions when possible. Any time you deviate from the standard install, you risk having problems that you can't easily solve.

I think I followed the official instructions step-by-step, but who knows because there are some steps that are not up to date with the latest version.

I believe that the OTA demo is the most difficult and I have not run it. I'm in the process of extracting the functionality from it because I feel that it is incompatible with the correct project structure for Fleet Provisioning and Device Shadow work (Jobs, etc).

So you don't have the OTA functionality yet? I've seen some other comment from you asking for a "full example" and I think this would be very interesting because anyone using AWS services will probably use more than one. I hope to have it working soon and add other production-level services like Fleet (it's basic!).

So I'll try the following workarounds:

  1. Compiling the OTA demo with VSCode.
  2. Compiling the MQTT demo with VSCode.
  3. Install 4.3.2 environment and start again (now that I have more knowledge about it).

Thank you and good luck!

SolidStateLEDLighting commented 2 years ago

I noticed that you have some of your installation on D: drive. C: drive is the default area for most Windows/Tool installs.

I repeat: "I'm in the process of extracting the functionality from it because I feel that it is incompatible with the correct project structure for Fleet Provisioning and Device Shadow work (Jobs, etc)."

I'm pulling down stream packages right now from the S3 bucket.... I still have to decode the cbor binary and then perform the remainder of the OTA. I'm not using the OTA demo directly. I'm reverse engineering from it.

K.

From: Aaron @.> Sent: Wednesday, January 5, 2022 6:06 PM To: espressif/esp-aws-iot @.> Cc: keith ssledlighting.com @.>; Mention @.> Subject: Re: [espressif/esp-aws-iot] Beta ?? Can we get the CMakeList and KConfig files? (CA-165) (Issue #73)

I was suggesting a reload of the window inside VS Code....

I'm using the "ESP-IDF 4.3 CMD" to open the console. I also use VSCode + ESP plugin, but I haven't used it with the demo projects.

Here is my best suggestion for a Windows build environment. Follow ALL the default installation instructions when possible. Any time you deviate from the standard install, you risk having problems that you can't easily solve.

I think I followed the official instructions step-by-step, but who knows because there are some steps that are not up to date with the latest version.

I believe that the OTA demo is the most difficult and I have not run it. I'm in the process of extracting the functionality from it because I feel that it is incompatible with the correct project structure for Fleet Provisioning and Device Shadow work (Jobs, etc).

So you don't have the OTA functionality yet? I've seen some other comment from you asking for a "full example" and I think this would be very interesting because anyone using AWS services will probably use more than one. I hope to have it working soon and add other production-level services like Fleet (it's basic!).

So I'll try the following workarounds:

  1. Compiling the OTA demo with VSCode.
  2. Compiling the MQTT demo with VSCode.
  3. Install 4.3.2 environment and start again (now that I have more knowledge about it).

Thank you and good luck!

— Reply to this email directly, view it on GitHubhttps://github.com/espressif/esp-aws-iot/issues/73#issuecomment-1005545844, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGOKE4UFRRFRZ2YGYR4LSDUUQJ2BANCNFSM5HPOZS7A. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.Message ID: @.***>