search

espressif / idf-component-manager

Tool for installing ESP-IDF components
https://components.espressif.com/
Apache License 2.0
44 stars 16 forks source link

idf.py fails verbosely and incomprehensibly without an internet connection (IDFGH-8169) (PACMAN-429) #8

Closed davidzuhn closed 2 years ago

davidzuhn commented 2 years ago

Answers checklist.

IDF version.

v5.1-dev-437-g5c1044d84d

Operating System used.

macOS

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

No response

What is the expected behavior?

I have a project that uses external dependencies (mdns).

When I try to do a build when I don't have an internet connection, I'd like to either a) be able to do my build with a warning about dependencies possibly being out of date, or b) get a concise & useful error message about needing an internet connection to check upstream repositories.

What is the actual behavior?

Instead I get this (from running idf.py):

Adding "set-target"'s dependency "fullclean" to list of commands with default set of options.
Executing action: fullclean
Executing action: set-target
Set Target to: esp32, new sdkconfig created. Existing sdkconfig renamed to sdkconfig.old.
Running cmake in directory /Users/zoo/esp32/esp-idf-latest-20220828/examples/get-started/hello_world/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DIDF_TARGET=esp32 -DCCACHE_ENABLE=0 /Users/zoo/esp32/esp-idf-latest-20220828/examples/get-started/hello_world"...
-- Found Git: /usr/bin/git (found version "2.32.1 (Apple Git-133)") 
-- The C compiler identification is GNU 11.2.0
-- The CXX compiler identification is GNU 11.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /Users/zoo/.espressif/tools/xtensa-esp32-elf/esp-2022r1-11.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Users/zoo/.espressif/tools/xtensa-esp32-elf/esp-2022r1-11.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/zoo/.espressif/tools/xtensa-esp32-elf/esp-2022r1-11.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32
Solving dependencies requirements
CMake Error at /Users/zoo/esp32/esp-idf-latest-20220828/tools/cmake/build.cmake:519 (message):
  Traceback (most recent call last):

    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
      conn = connection.create_connection(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/util/connection.py", line 72, in create_connection
      for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
    File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 954, in getaddrinfo
      for res in _socket.getaddrinfo(host, port, family, type, proto, flags):

  socket.gaierror: [Errno 8] nodename nor servname provided, or not known

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):

    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
      httplib_response = self._make_request(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
      self._validate_conn(conn)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
      conn.connect()
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
      self.sock = conn = self._new_conn()
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
      raise NewConnectionError(

  urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection
  object at 0x1023de250>: Failed to establish a new connection: [Errno 8]
  nodename nor servname provided, or not known

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):

    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
      resp = conn.urlopen(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 815, in urlopen
      return self.urlopen(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 815, in urlopen
      return self.urlopen(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 815, in urlopen
      return self.urlopen(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
      retries = retries.increment(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
      raise MaxRetryError(_pool, url, error or ResponseError(cause))

  urllib3.exceptions.MaxRetryError:
  HTTPSConnectionPool(host='api.components.espressif.com', port=443): Max
  retries exceeded with url: /components/espressif/mdns/ (Caused by
  NewConnectionError('<urllib3.connection.HTTPSConnection object at
  0x1023de250>: Failed to establish a new connection: [Errno 8] nodename nor
  servname provided, or not known'))

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):

    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_tools/api_client.py", line 158, in _base_request
      response = self.session.request(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
      resp = self.send(prep, **send_kwargs)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
      r = adapter.send(request, **kwargs)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/requests/adapters.py", line 565, in send
      raise ConnectionError(e, request=request)

  requests.exceptions.ConnectionError:
  HTTPSConnectionPool(host='api.components.espressif.com', port=443): Max
  retries exceeded with url: /components/espressif/mdns/ (Caused by
  NewConnectionError('<urllib3.connection.HTTPSConnection object at
  0x1023de250>: Failed to establish a new connection: [Errno 8] nodename nor
  servname provided, or not known'))

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):

    File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/prepare_components/__main__.py", line 3, in <module>
      main()
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/prepare_components/prepare.py", line 126, in main
      args.func(args)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/prepare_components/prepare.py", line 39, in prepare_dep_dirs
      ComponentManager(args.project_dir).prepare_dep_dirs(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/core.py", line 404, in prepare_dep_dirs
      downloaded_component_paths = download_project_dependencies(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/dependencies.py", line 99, in download_project_dependencies
      solution = solver.solve()
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/version_solver/version_solver.py", line 29, in solve
      self.solve_manifest(manifest)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/version_solver/version_solver.py", line 48, in solve_manifest
      self.solve_component(requirement)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_manager/version_solver/version_solver.py", line 51, in solve_component
      cmp_with_versions = requirement.source.versions(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_tools/sources/web_service.py", line 95, in versions
      cmp_with_versions = self.api_client.versions(name, spec, target)
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_tools/api_client.py", line 196, in versions
      body = self._base_request(
    File "/Users/zoo/.espressif/python_env/idf5.1_py3.9_env/lib/python3.9/site-packages/idf_component_tools/api_client.py", line 178, in _base_request
      raise APIClientError('HTTP request error')

  idf_component_tools.api_client_errors.APIClientError: HTTP request error

Call Stack (most recent call first):
  /Users/zoo/esp32/esp-idf-latest-20220828/tools/cmake/project.cmake:434 (idf_build_process)
  CMakeLists.txt:6 (project)

-- Configuring incomplete, errors occurred!
See also "/Users/zoo/esp32/esp-idf-latest-20220828/examples/get-started/hello_world/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1, output of the command is in the /Users/zoo/esp32/esp-idf-latest-20220828/examples/get-started/hello_world/build/log/idf_py_stderr_output_62008 and /Users/zoo/esp32/esp-idf-latest-20220828/examples/get-started/hello_world/build/log/idf_py_stdout_output_62008

which is neither concise nor particularly helpful unless one happens to be good at deciphering Python stack traces (which is generally not a skill I expect to be needed to be an ESP32 developer).

Steps to reproduce.

  1. cd $IDF_PATH/examples/get-started/hello_world

  2. add a dependency on an external component put this content into main/idf_component.yml

    ## IDF Component Manager Manifest File
dependencies:
espressif/mdns: "==1.0.5"
## Required IDF version
idf:
version: ">=4.1.0"
# # Put list of dependencies here
# # For components maintained by Espressif:
# component: "~1.0.0"
# # For 3rd party components:
# username/component: ">=1.0.0,<2.0.0"
# username2/component2:
#   version: "~1.0.0"
#   # For transient dependencies `public` flag can be set.
#   # `public` flag doesn't have an effect dependencies of the `main` component.
#   # All dependencies of `main` are public by default.
#   public: true

  3. Disable your internet access

  4. Run 'idf.py set-target esp32'

Build or installation Logs.

No response

More Information.

I'd like to see something catch all of those exceptions and print a single error message.

In the case where the dependency is found locally, it might be nice to have an option where the build continues without checking against the upstream repository (airplane mode development).

kumekay commented 2 years ago

@davidzuhn Thank you for the issue, we will improve reporting for the situation.

When all your dependencies are already downloaded, regular rebuild doesn't try to connect even now. However, changing of target discards information about dependencies and it will try to check for versions available for the new target. And it's hard to change this behaviour now.