How to generate firmware package? /project_build_config.json: no such file or directory

[paclema]

Hello,

I am trying to upload the firmware package to esp-insights dashboard but it says that something went wrong:

I am using Arduino and platformio to build the firmware, so I have created the zip file using also the build/ directory from platformio, which contains the firmware.bin, elf, map and the rest of the files but it looks that this "project_build_config.json" file is not generated using arduino framework for firmware compilation.

If I use the ESP-IDF hello-world example and compile it, it also does not create such as "project_build_config.json" but it creates a "project_description.json" file. If I use that file (change the respective _appelf and _appbine properties to the correct ones) and upload this also within the zip file, the error I get while uploading the zip file is:

• So what are the exact files (bin, elf, map, what else?) that should be added to the firmware package zip file?
• How can I adjust the "project_description.json" file or what are the json properties that the backend needs?

Thank you!

[shahpiyushv]

@paclema , just creating a zip won't help if Insights itself is not enabled. Have you followed the various READMEs under this project, especially the top level README, the examples README and the minimal diagnostics README?

After you have integrated the ESP Insights components and functions in your application, the zip package is automatically created in the build process, if used with esp idf. Similar may not be created in platform io, so you may create one manually. Sample contents of the package:

minimal_diagnostics-v1.0/
├── bootloader
│   └── bootloader.bin
├── minimal_diagnostics.bin
├── minimal_diagnostics.elf
├── minimal_diagnostics.map
├── partition_table
│   └── partition-table.bin
├── partitions.csv
├── project_build_config.json
├── project_description.json
└── sdkconfig

The project_build_config.json has information of all esp-idf submodules and the project name and versionn. You can create your own file with only this populated too

{
    "project": {
        "name": "minimal_diagnostics",
        "version": "1.0"
    }
}

Note that the project name and version should match what you have in your CMakefile.txt . The same are also used in the file name <project_name>-<version>.zip

[paclema]

Thank you @shahpiyushv ! I could prepare the zip file following your tips and now I can also upload it successfully.

However, there must be a mismatch with the project version or something because the esp-insights logs can not be totally shown since it does not find the correct firmware package version:

As you can see the esp-insights works quite well. I can see that there are logs coming, also custom metrics and variables work well but I am still not sure what do I do wrong by uploading the package.

On one hand, the node appears in the dashboard with the next configuration:

Also checking the esp32 App Version API the project version is 3.7.0 too. I can print it using the esp32 Serial monitor the same information:

But in the node info, the version appears with an extra "v" that kind of confuses me of what is the real version that esp-insights is trying to find:

As you can see in the firmware package sections, I have uploaded several packages with different version names, some combinations also match the ones described in the node list section:

I have followed your indications when building the zip file, using platformio build to get the generated bin, elf and map files but with my specific test name:

ControlDevice-3.7.0/
├── bootloader
│   └── bootloader.bin
├── ControlDevice-3.7.0.bin
├── ControlDevice-3.7.0.elf
├── ControlDevice-3.7.0.map
├── partition_table
│   └── partition-table.bin
├── partitions.csv
├── project_build_config.json
├── project_description.json
└── sdkconfig

I have tried using both ControlDevice-3.7.0 and ControlDevice-v3.7.0 in combination of "version": "3.7.0" and "version": "v3.7.0" for the project_build_config.json:

{
    "project": {
        "name": "ControlDevice",
        "version": "3.7.0"
    }
}

Why the node logs entries are still saying that I must upload the firmware package? or how can I be sure which is the firmware package version that should be matching with the node?

I am uploading the firmware package using insights dashboard because from rainmaker dashboard it appears that: You are not authorized to perform this operation

Thank you for your help!

[shahpiyushv]

@paclema , every time you compile the same project, even without any change, it generates a new binary and elf. The Insights backend can map correctly only if the exact same binary that was flashed on the board is added into the package (along with the corresponding elf, map, etc.). Note that even an idf.py flash command recreates a new bin.

[paclema]

@shahpiyushv yes you are right, and every time that I change the version from 3.7.0 to v3.7.0 I have rebuild all the binaries and so on, but the issue persists :S

[shahpiyushv]

Also, I think I made a small mistake in my comment above. The zip file name should be <project_name>-v<version>.zip. Notice the extra v indicating version. The CMakeLists.txt would just have the version without v. While uploading, do not manually enter any version. It will be fetched from the files automatically.

[vikramdattu]

Hi @paclema to add to the Piyush...

The info is resolved from the project .elf file (this is a part of the firmware package created by esp-insights). Cloud checks if the checksum of any uploaded elfs(via firmware packages) matches with the one provided by the message.

Even if you make a small change in your code, the elf and the checksum generated are different and hence the package needs to be uploaded again for the same.

Hence, here for the resolution of the info, the name ControlDevice-3.7.0.elf has no importance but the contents of the elf itself have. (The checksum(sha256) is part of esp_app_desc_t structure which is at the start of from segment of the image).

[paclema]

Thank you a lot @shahpiyushv and @vikramdattu for your input!

It took some time but I have come up with some working example integrating this component with arduino-esp32 framework. It requires some changes in the platform to integrate this elf hash for arduino framework (I have created a PR for that https://github.com/platformio/platform-espressif32/pull/973 ), and also to integrate the idf component into the arduino-esp32 framework by itself (I have created a PR for that matter https://github.com/espressif/esp32-arduino-lib-builder/pull/99)

I have also created a platformio script to build up the zip package containing the data that you guys have kindly described and verified that it works well. You have here the example: ESP-Insights with arduino-esp32 framework example

You can also see more information in the PR https://github.com/platformio/platform-espressif32/pull/973#issuecomment-1337751499.