Internet for offline installer

Yurbaan commented 1 year ago

Hello everyone! I encountered such an error when using esp-idf 5.0.1 offline installer (for Windows). Do I understand correctly that an internet connection is needed for offline installation?:)

                          Downloading https://github.com/espressif/esp-rom-elfs/releases/download/20220823/esp-rom-elfs-20220823.tar.gz
                          Destination: C:\Espressif\dist\esp-rom-elfs-20220823.tar.gz.tmp
                          WARNING: Download failure: <urlopen error [Errno 11001] getaddrinfo failed>
                          WARNING: Failed to download https://github.com/espressif/esp-rom-elfs/releases/download/20220823/esp-rom-elfs-20220823.tar.gz to C:\Espressif\dist\esp-rom-elfs-20220823.tar.gz.tmp
                          Downloading https://github.com/espressif/esp-rom-elfs/releases/download/20220823/esp-rom-elfs-20220823.tar.gz
                          Destination: C:\Espressif\dist\esp-rom-elfs-20220823.tar.gz.tmp
                          WARNING: Download failure: <urlopen error [Errno 11001] getaddrinfo failed>
                          WARNING: Failed to download https://github.com/espressif/esp-rom-elfs/releases/download/20220823/esp-rom-elfs-20220823.tar.gz to C:\Espressif\dist\esp-rom-elfs-20220823.tar.gz.tmp
                          Downloading https://github.com/espressif/esp-rom-elfs/releases/download/20220823/esp-rom-elfs-20220823.tar.gz
                          Destination: C:\Espressif\dist\esp-rom-elfs-20220823.tar.gz.tmp
                          WARNING: Download failure: <urlopen error [Errno 11001] getaddrinfo failed>
                          WARNING: Failed to download https://github.com/espressif/esp-rom-elfs/releases/download/20220823/esp-rom-elfs-20220823.tar.gz to C:\Espressif\dist\esp-rom-elfs-20220823.tar.gz.tmp
                          ERROR: Failed to download, and retry count has expired
                          Please make sure you have a working Internet connection.
                          By downloading and using the offline installer from https://dl.espressif.com/dl/esp-idf you might be able to work around this issue.

Log after error

liux-pro commented 1 year ago

Same issue ! It happend several months ago, #159 , this happend again now. Please fix it.

I have two suggests for the developer of this offline installer.

Test offline installer on a offline PC for any new version.
We need a version number to each installer. For example ,you fix this issue now, the version number still be 5.0.1, users can't see difference, it do have some changes.

georgik commented 1 year ago

Thank you for reporting the problem.

We're investigating why those tools were not downloaded during build process.

Also CI scripts for Offline testing in Hyper-V and Azure will be added.

georgik commented 1 year ago

Note from research: It's possible to turn off the network even in Windows Sandbox

georgik commented 1 year ago

Thank you @Yurbaan for pointing at the problem. We found that also some other files from tools directory were not included. We yanked the 5.0.1 release and re-run the build with proper configuration to include also missing files. The new rebuild of installer is available at https://github.com/espressif/idf-installer/releases/tag/offline-5.0.1 Please, test the updated version.

georgik commented 1 year ago

The installation now works without problem. The build is failing when the computer is Offline:

CMake Error at C:/Espressif/frameworks/esp-idf-v5.0.1/tools/cmake/build.cmake:519 (message):
  ERROR: Cannot establish a connection to the component registry.  Are you
  connected to the internet?

@kumekay Is there a way to tell the build system to avoid downloading files or give it an instruction to pre-cache registry and store it to Offline installer.

When internet connection is enabled the build continues. It seems that resolution is required to get lock file in blink example:

Solving dependencies requirements
.......Updating lock file at C:/Espressif/frameworks/esp-idf-v5.0.1/examples/get-started/blink\dependencies.lock
Processing 2 dependencies:
[1/2] espressif/led_strip (2.3.1)

Also notice, incorrect path. It's mixed backslash and slash.

kumekay commented 1 year ago

@georgik

The component manager supports two essential features required for the desired behavior: it can utilize a registry mirror, and the registry mirror can be a local directory. Hypothetically, we could include a partial registry mirror in the offline installer.

However, some limitations need to be addressed first:

There is no straightforward method for creating a partial mirror of the registry, containing only components used in examples.
The component manager expects a full mirror and will attempt to fetch all required components for a project from it. Consequently, if you add components to your project that are not available in the partial mirror, you must switch to a full mirror.
Components used in examples have independent development cycles and are not strictly tied to specific ESP-IDF versions. Relying on an offline mirror could lead to the usage of dependencies with bugs and security vulnerabilities.

We must first determine how to mitigate these issues and ensure that the behavior in offline mode is not confusing.

P.S.: I have fixed the mixed slashes issue, and it will be included in the next release of the component manager

georgik commented 1 year ago

Thank you @kumekay for the explanation.

Basically, examples which are using components introduced in 5.x requires connection to the internet in build time.

The build with components could work offline to some extend. For this release, we declare it as a limitation. The limitation might be removed in future releases after updating the implementation of Component Manager.

The installation works completely offline and installer is not failing.

espressif / idf-installer

Internet for offline installer #182