Builds fail without Internet access (IDFGH-12120) (PACMAN-808)

readmodifywrite commented 7 months ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

release/5.1 8d2dbd461fc7a31b3d3e55ec12149e53130c13f6

Operating System used.

Linux

How did you build your project?

Command line with CMake

If you are using Windows, please specify command line type.

None

What is the expected behavior?

Builds should succeed without an Internet connection.

export.sh is checking the component registry, and failing if there is no Internet available.

Builds should never be updating components on their own, so there is no need for this check at all. Firmware builds should succeed without Internet connectivity.

What is the actual behavior?

Builds fail without an Internet connection.

Steps to reproduce.

Build an ESP32 project without Internet access.

Build or installation Logs.

jeremy@Saturn ~/J/T/f/p/esp32_firmware (main)> ./build.sh
~/JEREMY/Twine/firmware/pickup_firmware/esp-idf ~/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware
Setting IDF_PATH to '/home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf'
Detecting the Python interpreter
Checking "python3" ...
Python 3.8.10
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Constraint file: /home/jeremy/.espressif/espidf.constraints.v5.1.txt
Requirement files:
 - /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /home/jeremy/.espressif/python_env/idf5.1_py3.8_env/bin/python
Python requirements are satisfied.
Added the following directories to PATH:
  /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/espcoredump
  /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/partition_table
  /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/app_update
  /home/jeremy/.espressif/tools/xtensa-esp-elf-gdb/12.1_20221002/xtensa-esp-elf-gdb/bin
  /home/jeremy/.espressif/tools/riscv32-esp-elf-gdb/12.1_20221002/riscv32-esp-elf-gdb/bin
  /home/jeremy/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin
  /home/jeremy/.espressif/tools/xtensa-esp32s2-elf/esp-12.2.0_20230208/xtensa-esp32s2-elf/bin
  /home/jeremy/.espressif/tools/xtensa-esp32s3-elf/esp-12.2.0_20230208/xtensa-esp32s3-elf/bin
  /home/jeremy/.espressif/tools/riscv32-esp-elf/esp-12.2.0_20230208/riscv32-esp-elf/bin
  /home/jeremy/.espressif/tools/esp32ulp-elf/2.35_20220830/esp32ulp-elf/bin
  /home/jeremy/.espressif/tools/openocd-esp32/v0.12.0-esp32-20230419/openocd-esp32/bin
  /home/jeremy/.espressif/python_env/idf5.1_py3.8_env/bin
  /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

~/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware
~/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/build ~/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware
-- Building ESP-IDF components for target esp32c3
-- Checking Python dependencies...
Python requirements are satisfied.
Constraint file: /home/jeremy/.espressif/espidf.constraints.v5.1.txt
Requirement files:
 - /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /home/jeremy/.espressif/python_env/idf5.1_py3.8_env/bin/python
CMake Error at /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/tools/cmake/build.cmake:540 (message):
  ERROR: Cannot establish a connection to the component registry.  Are you
  connected to the internet?

Call Stack (most recent call first):
  /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/tools/cmake/project.cmake:547 (idf_build_process)
  CMakeLists.txt:14 (project)

-- Configuring incomplete, errors occurred!
See also "/home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/build/CMakeFiles/CMakeOutput.log".

More Information.

No response

kumekay commented 7 months ago

Hello @readmodifywrite

Please help to provide some information about your information and the project state.

Please check the version of the Component manager. Run compote version or, if it fails python -m idf_component_manager --help in the same terminal where you run idf.py to get the version.
Do you have any dependencies specified in idf_component.yml files in any of the components in the project? If so,
Do you have dependencies.lock and the managed component directory in your project?
Do you have any URLs defined in ~/.espressif/idf_component_manager.yml or any env variables starting with IDF_COMPONENT_ defined?

readmodifywrite commented 7 months ago

compote version: 1.4.1
There is a dependencies.lock, see attached (with .txt added for Github). The versions are pinned so there shouldn't be any need to go out to the Internet.
There is no idf_component_manager.yml in ~/.espressif

dependencies.lock.txt

kumekay commented 7 months ago

@readmodifywrite Thank you for your responses,

Do you have espressif__mdns directory in mananged_components of your project?
Did you change the target or ESP-IDF version since the last successful CMake run?

Could you please update the component manager by running pip install -U idf-component-manager in your ESP-IDF shell? We improved network related error messages recently.

readmodifywrite commented 7 months ago

Yes, it is in managed_components.

No changes to target or IDF version, when I filed this report.

However, I am now experimenting with v5.2. I will update if this issue persists.

I cannot run the pip command now that I've updated to 5.2: jeremy@Saturn:~/JEREMY/Twine/firmware/pickup_firmware/esp-idf$ pip3 install -U idf-component-manager bash: /home/jeremy/.espressif/python_env/idf5.2_py3.8_env/bin/pip3: /home/jeremy/.espressif/python_env/idf5.1_py3.8_env/bin/python: bad interpreter: No such file or directory

See this issue: https://github.com/espressif/esp-idf/issues/13196 5.2 screws up the Python venv. I managed to work around it to get builds working, but it looks like that breaks other things that are less obvious to fix.

readmodifywrite commented 7 months ago

So, to work around all of this, would it not make more sense to just directly copy the MDNS source into my project, and get rid of the component manager entirely?

I really don't see the value of the component manager at all, especially since it is creating more problems than it solves.

We never want components to auto update during a build in the embedded world.

Why not just put MDNS in the esp-idf/components like everything else? The component manager is just extra overhead for nothing. What is the actual use case here?

readmodifywrite commented 7 months ago

Ok, so you can't just manually include the MDNS source in your project, even after also adding the Kconfig to get the #defines for the configuration. It will still break with a ton of random errors, see below.

This whole component manager thing seems to be designed to be impossible to get rid of.

I want to manually build mDNS in my project. I do not want to use the component manager (none of the handful of other "components" are of any use to me anyway). I have no reason to deal with this extra baggage in my builds.

How do I get a component to build manually in my project? The component manager is doing some kind of extra processing that doesn't appear to be documented anywhere.

Build error log below:

In file included from /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c:12: /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c: In function '_udp_recv': /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:265:27: error: format '%d' expects argument of type 'int', but argument 7 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=] 265 | #define LOG_COLOR(COLOR) "\033[0;" COLOR "m" | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:268:27: note: in expansion of macro 'LOG_COLOR' 268 | #define LOG_COLOR_E LOG_COLOR(LOG_COLOR_RED) | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:282:37: note: in expansion of macro 'LOG_COLOR_E' 282 | #define LOG_FORMAT(letter, format) LOGCOLOR ## letter #letter " (%" PRIu32 ") %s: " format LOG_RESET_COLOR "\n" | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:410:86: note: in expansion of macro 'LOG_FORMAT' 410 | if (level==ESP_LOG_ERROR ) { esp_log_write(ESP_LOG_ERROR, tag, LOG_FORMAT(E, format), esp_log_timestamp(), tag, ##VA_ARGS); } \ | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:432:41: note: in expansion of macro 'ESP_LOG_LEVEL' 432 | if ( LOG_LOCAL_LEVEL >= level ) ESP_LOG_LEVEL(level, tag, format, ##VA_ARGS); \ | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:340:38: note: in expansion of macro 'ESP_LOG_LEVEL_LOCAL' 340 | #define ESP_LOGE( tag, format, ... ) ESP_LOG_LEVEL_LOCAL(ESP_LOG_ERROR, tag, format, ##VA_ARGS) | ^~~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:29: note: in expansion of macro 'ESP_LOGE' 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE__, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c:146:13: note: in expansion of macro 'HOOK_MALLOC_FAILED' 146 | HOOK_MALLOC_FAILED; | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:265:27: error: format '%d' expects argument of type 'int', but argument 7 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=] 265 | #define LOG_COLOR(COLOR) "\033[0;" COLOR "m" | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:269:27: note: in expansion of macro 'LOG_COLOR' 269 | #define LOG_COLOR_W LOG_COLOR(LOG_COLOR_BROWN) | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:282:37: note: in expansion of macro 'LOG_COLOR_W' 282 | #define LOG_FORMAT(letter, format) LOGCOLOR ## letter #letter " (%" PRIu32 ") %s: " format LOG_RESET_COLOR "\n" | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:411:86: note: in expansion of macro 'LOG_FORMAT' 411 | else if (level==ESP_LOG_WARN ) { esp_log_write(ESP_LOG_WARN, tag, LOG_FORMAT(W, format), esp_log_timestamp(), tag, ##VA_ARGS); } \ | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:432:41: note: in expansion of macro 'ESP_LOG_LEVEL' 432 | if ( LOG_LOCAL_LEVEL >= level ) ESP_LOG_LEVEL(level, tag, format, ##VA_ARGS); \ | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:340:38: note: in expansion of macro 'ESP_LOG_LEVEL_LOCAL' 340 | #define ESP_LOGE( tag, format, ... ) ESP_LOG_LEVEL_LOCAL(ESP_LOG_ERROR, tag, format, ##__VA_ARGS) | ^~~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:29: note: in expansion of macro 'ESP_LOGE' 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c:146:13: note: in expansion of macro 'HOOK_MALLOC_FAILED' 146 | HOOK_MALLOC_FAILED; | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:101: error: format '%d' expects argument of type 'int', but argument 7 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=] 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:282:59: note: in definition of macro 'LOG_FORMAT' 282 | #define LOG_FORMAT(letter, format) LOGCOLOR ## letter #letter " (%" PRIu32 ") %s: " format LOG_RESET_COLOR "\n" | ^~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:432:41: note: in expansion of macro 'ESP_LOG_LEVEL' 432 | if ( LOG_LOCAL_LEVEL >= level ) ESP_LOG_LEVEL(level, tag, format, ##VA_ARGS__); \ | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:340:38: note: in expansion of macro 'ESP_LOG_LEVEL_LOCAL' 340 | #define ESP_LOGE( tag, format, ... ) ESP_LOG_LEVEL_LOCAL(ESP_LOG_ERROR, tag, format, ##VA_ARGS) | ^~~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:29: note: in expansion of macro 'ESP_LOGE' 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c:146:13: note: in expansion of macro 'HOOK_MALLOC_FAILED' 146 | HOOK_MALLOC_FAILED; | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:101: error: format '%d' expects argument of type 'int', but argument 7 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=] 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:282:59: note: in definition of macro 'LOG_FORMAT' 282 | #define LOG_FORMAT(letter, format) LOGCOLOR ## letter #letter " (%" PRIu32 ") %s: " format LOG_RESET_COLOR "\n" | ^~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:432:41: note: in expansion of macro 'ESP_LOG_LEVEL' 432 | if ( LOG_LOCAL_LEVEL >= level ) ESP_LOG_LEVEL(level, tag, format, ##VA_ARGS); \ | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:340:38: note: in expansion of macro 'ESP_LOG_LEVEL_LOCAL' 340 | #define ESP_LOGE( tag, format, ... ) ESP_LOG_LEVEL_LOCAL(ESP_LOG_ERROR, tag, format, ##__VA_ARGS) | ^~~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:29: note: in expansion of macro 'ESP_LOGE' 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c:146:13: note: in expansion of macro 'HOOK_MALLOC_FAILED' 146 | HOOK_MALLOC_FAILED; | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:265:27: error: format '%d' expects argument of type 'int', but argument 7 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=] 265 | #define LOG_COLOR(COLOR) "\033[0;" COLOR "m" | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:270:27: note: in expansion of macro 'LOG_COLOR' 270 | #define LOG_COLOR_I LOG_COLOR(LOG_COLOR_GREEN) | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:282:37: note: in expansion of macro 'LOG_COLOR_I' 282 | #define LOG_FORMAT(letter, format) LOGCOLOR ## letter #letter " (%" PRIu32 ") %s: " format LOG_RESET_COLOR "\n" | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:414:86: note: in expansion of macro 'LOG_FORMAT' 414 | else { esp_log_write(ESP_LOG_INFO, tag, LOG_FORMAT(I, format), esp_log_timestamp(), tag, ##VA_ARGS); } \ | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:432:41: note: in expansion of macro 'ESP_LOG_LEVEL' 432 | if ( LOG_LOCAL_LEVEL >= level ) ESP_LOG_LEVEL(level, tag, format, ##VA_ARGS); \ | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:340:38: note: in expansion of macro 'ESP_LOG_LEVEL_LOCAL' 340 | #define ESP_LOGE( tag, format, ... ) ESP_LOG_LEVEL_LOCAL(ESP_LOG_ERROR, tag, format, ##VA_ARGS) | ^~~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/private_include/mdns_private.h:162:29: note: in expansion of macro 'ESP_LOGE' 162 | #define HOOK_MALLOC_FAILED ESP_LOGE(TAG, "Cannot allocate memory (line: %d, free heap: %d bytes)", LINE, esp_get_free_heap_size()); | ^~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns_networking_lwip.c:146:13: note: in expansion of macro 'HOOK_MALLOC_FAILED' 146 | HOOK_MALLOC_FAILED; | ^~~~~~ In file included from /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns.c:13: /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp32_firmware/main/mdns/mdns.c: In function '_mdns_mangle_name': /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:265:27: error: format '%d' expects argument of type 'int', but argument 7 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=] 265 | #define LOG_COLOR(COLOR) "\033[0;" COLOR "m" | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:268:27: note: in expansion of macro 'LOG_COLOR' 268 | #define LOG_COLOR_E LOG_COLOR(LOG_COLOR_RED) | ^~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:282:37: note: in expansion of macro 'LOG_COLOR_E' 282 | #define LOG_FORMAT(letter, format) LOGCOLOR ## letter #letter " (%" PRIu32 ") %s: " format LOG_RESET_COLOR "\n" | ^~~~~~ /home/jeremy/JEREMY/Twine/firmware/pickup_firmware/esp-idf/components/log/include/esp_log.h:410:86: note: in expansion of macro 'LOG_FORMAT' 410 | if (level==ESP_LOG_ERROR ) { esp_log_write(ESP_LOG_ERROR, tag, LOG_FORMAT(E, format), esp_log_timestamp(), tag, ##VA_ARGS); } \ | ^~~~~~

.... truncated.

It is over 6000 lines of this nonsense.

readmodifywrite commented 7 months ago

How to fix the build: Add: component_compile_options(-Wno-error=format= -Wno-format) to CMakeLists.txt.

Add the component Kconfig to the project root to get the #defines and menuconfig.

I can confirm that moving the managed component and compiling the source directly fixes this issue.

igrr commented 7 months ago

I want to manually build mDNS in my project. I do not want to use the component manager (none of the handful of other "components" are of any use to me anyway). I have no reason to deal with this extra baggage in my builds.

In this case, the simplest option is to put the mdns component from https://github.com/espressif/esp-protocols/tree/master/components/mdns (or a specific version in that repo) into the components subdirectory of your project.

(The component manager helps with dependency hierarchies more than one level deep, but in your case simply getting the single component you need manually should work.)

kumekay commented 7 months ago

Hello @readmodifywrite

Some components are moved out from the ESP-IDF to the component registry to deliver bug fixes and new features, not waiting for the next ESP-IDF release. If you would like to download a single component, you can either clone the repo, as Ivan suggested. Or download an archive from the registry (download archive link in the right-hand bar) and unpack to the components directory of the project.

If you want to turn off the component manager entirely, you can set the environment variable IDF_COMPONENT_MANAGER to 0. https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/tools/idf-component-manager.html?highlight=idf_component_manager#disabling-the-component-manager

kumekay commented 7 months ago

Hello @readmodifywrite I accidentally closed this issue yesterday,

We never want components to auto update during a build in the embedded world.

Neither we do! Now, the dependency solver may run on build in a few cases:

No dependencies.lock file.
Some requirements have been modified in the idf_component.yml manifest files
The target has changed.
The IDF version has been updated.

If your case doesn't fall into these categories, it's likely a bug. We'd greatly appreciate your assistance in diagnosing this issue further. Please provide us with the full output of ⁣idf.py reconfigure using the latest component manager version (1.5.1) and any ESP-IDF version, if you can still reproduce the problem. And thank you for your collaboration and patience.

Running of solver when the target or IDF version has changed is a necessity because dependencies may be incompatible. We are now working on the fix to only download something when deps are actually incompatible. So likely this will help in your case.

We really want to make a useful tool, not an annoying one, if you wish to help the whole community by giving us more data, it will be very kind from your side.

jgeertsema commented 6 months ago

I found this post after wondering why I couldn't build without internet due to the same "ERROR: Cannot establish a connection to the component registry. Are you connected to the internet?"

I've noticed that hitting the ESP-IDF: Full Clean will require an internet connection before it can build again. Is this the expected behavior, as it wasn't listed as one of the four reason listed above?

kumekay commented 6 months ago

@jgeertsema I apologize for the confusion. The logic behind it is simple: fullclean command removes most of the files created by the build process. Because the managed_components directory is created during it, the directory is also removed.

During the next run of CMake, it will be recreated. If you already have all the required components in the system-wide cache, no HTTP requests will be performed. You can find the location of the cache by running compote cache path.

A couple of exceptions are sdkconfig and dependencies.lock not removed by full clean.

In which scenario it causes you problems? We will find the best workaround.

jgeertsema commented 6 months ago

I believe this part isn't working as intended.

If you already have all the required components in the system-wide cache, no HTTP requests will be performed. You can find the location of the cache by running compote cache path.

In my project I have 1 managed component that requires version 1.12.1. If I navigate to the cache path I can see the managed component at version 1.12.1. After running fullclean the 1.12.1 managed component is still in the cache folder. However if I try to build the project without internet it will still get the error: "ERROR: Cannot establish a connection to the component registry. Are you connected to the internet?"

If I'm understanding correctly because I already have version 1.12.1 in my cache I should be able to build after a fullclean without an internet connection.

kumekay commented 6 months ago

@jgeertsema Which version of the component manager are you using? I've tried with the most recent one - v1.5.2 and I wasn't able to reproduce the issue. We had a regression some time ago, though. Could you please try to update? https://docs.espressif.com/projects/idf-component-manager/en/latest/guides/updating_component_manager.html

jgeertsema commented 6 months ago

@kumekay I was using version v1.4.1 and I updated it before my last post to version v1.5.2. I was still having the issue when I made the post after updating, but today I can not recreate the issue either. I now see this "HINT: Cannot establish a connection to the component registry. Skipping checks of dependency changes." and it continues the build like I would expect.

kumekay commented 6 months ago

@jgeertsema

HINT: Cannot establish a connection to the component registry. Skipping checks of dependency changes.

We have a check for the fact if the component was deleted from the registry, and this message from this checker. We are reworking now how we deal with online/offline modes, and the behavior should be much clearer in the next release.

jonahbron commented 6 months ago

I also am butting into this issue. In my particular case, I'm trying to build a firmware with ESP-IDF in a Nix environment, but a Nix derivation has no network connection.

@kumekay I can run the build with IDF_COMPONENT_MANAGER=0, but then the dependencies aren't included properly, even if they're already installed to managed_components/ from a previous run. But if I run with the component manager enabled, then it fails due to the network not being available.

At the very least, if dependencies are already present, the network call should fail silently and non-fatally.

It would be ideal to have a non-mutating separate command to run that installs the dependencies specified by the lock file. That needs to be a separate action from "update" or "build".

kumekay commented 5 months ago

@jonahbron We will change the default behavior in the next feature release

hsnprojects commented 5 months ago

I am having the same issue that I cannot build in VSCODE/ESPIDF if the machine has no access to the component registry. Auto updating components is not a good idea, and I would rather trust developer's knowledge about the components they use rather than going out of your way to make it harder to everyone.

kumekay commented 4 months ago

@hsnprojects It should be fixed in 2.0, you can try this commit: b543e18dd76df29d51a9a481928519ff3a031951 now

https://github.com/espressif/idf-component-manager?tab=readme-ov-file#installing-a-development-version-of-the-component-manager

hfudev commented 1 month ago

closing the issue since 2.0.0 has been released.

espressif / idf-component-manager