How to place the downloaded components in the specified directory. (PACMAN-526)

[WangYuxin-esp]

The Component Manager version

v1.2.1

ESP-IDF Version

v4.4.1

python Version

3.6.9

Operating System

ubuntu

Browser (for https://components.espressif.com Issues)

Chrome

Description

Many of my projects depend on the same component. How do I write the .yml file for each example to download the corresponding components to a unified directory.

To Reproduce

None.

Expected behaviour

My current directory structure is as follows:

my_components
example1
    +main
         +CMakeLists.txt 
         +idf_component.yml
    +CMakeLists.txt 
example2
    +main
         +CMakeLists.txt 
         +idf_component.yml
    +CMakeLists.txt 
...

I hope that each example can share the same my_components/esp-button, instead of downloading the esp-button component to the managed_components directory of each example.

Additional info

No response

I have checked existing issues and online Documentation

• [X] I confirm I have checked existing issues and online Documentation.

[kumekay]

Hi @WangYuxin-esp thank you for the question,

There is a field override_path you can use in example's idf_component.yml

i.e. in example1/main/idf_component.yml you can specify dependency as:

dependencies:
  WangYuxin/component_1:
    version: "^1.0.0"
    override_path: ../../my_components/component_1

One example: https://github.com/espressif/example_components/blob/main/cmp/examples/cmp_ex/main/idf_component.yml#L5

Some documentation on the topic: https://github.com/espressif/idf-component-manager/wiki#add-examples-to-the-component

Is it what you are looking for?

[WangYuxin-esp]

@kumekay Thank you for quick reply! However, the current field override_path does not seem to meet my needs. What I need is to establish a global component for multiple examples. When running one of these examples for the first time, the compilation system will download the component I need from github. Then, when running other examples, the compilation system only checks whether there are corresponding components in the corresponding directory.

[kumekay]

@WangYuxin-esp Unfortunately, what you are asking is quite hard to implement.

There are two different approaches to how dependency management systems store dependencies.

• Either system-wide (what you are looking for), for example: go lang, ruby gems
• Per-project: rust cargo, js npm

Both approaches have pros and cons, but the component manager follows the second one. And in most cases, it makes the developer's life easier. Usually, components are not big, so not much disk space is used.

Even though components are stored per project, it doesn't mean that they are downloaded every time you need them. Once downloaded, they are stored in a system-wide cache. A source repository still will be touched every time the dependencies.lock file is generated to refresh branches, but a complete clone is performed only once.

Could you please explain in detail your concerns about per-project dependency installations? So we will try to address them.

[xcguang]

@kumekay If I use path field, it will raise an error in my manifest as following, if removed it, works well.

## IDF Component Manager Manifest File
dependencies:
  idf:
    version: ">=5.0.0"

  esp32-camera:
    version: "*"
    git: "https://github.com/espressif/esp32-camera.git"
    path: "../../camera_components/"

the error:

File "/Users/xuchunguang/.espressif/python_env/idf5.0_py3.9_env/lib/python3.9/site-packages/idf_component_tools/git_client.py", line 183, in run
      raise GitCommandError(

  idf_component_tools.git_client.GitCommandError: 'git --work-tree
  /var/folders/mp/2wt44l1s7cqfjmt02nmkhnn00000gn/T/tmpxmux1j6z --git-dir
  /Users/xuchunguang/Library/Caches/Espressif/ComponentManager/b_git_55c49609
  checkout --force 402b811b835cd348343b567a97fdf984c9d16fb9 --
  ../../camera_components/' failed with exit code 128

  fatal: ../../camera_components/：'../../camera_components/' 在位于
  '/private/var/folders/mp/2wt44l1s7cqfjmt02nmkhnn00000gn/T/tmpxmux1j6z'
  的仓库之外

Call Stack (most recent call first):
  /Users/xuchunguang/workspace/project/esp-iot-bridge/esp-idf/tools/cmake/project.cmake:440 (idf_build_process)
  CMakeLists.txt:16 (project)

-- Configuring incomplete, errors occurred!
See also "/Users/xuchunguang/workspace/project/esp-iot-solution/examples/camera/pic_server/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1, output of the command is in the /Users/xuchunguang/workspace/project/esp-iot-solution/examples/camera/pic_server/build/log/idf_py_stderr_output_47656 and /Users/xuchunguang/workspace/project/esp-iot-solution/examples/camera/pic_server/build/log/idf_py_stdout_output_47656

Could you give any advice on this?

[kumekay]

@xcguang, the path field used together with git has a special meaning. It can be used to specify a directory inside a repo for the case when a component is not in the root of the repo.

If you want to use a component already existing on your machine, you can use a local directory instead.

dependencies:
  esp32-camera:
    path: "../../camera_components/"

Won't a component from the registry be a better option for you? https://components.espressif.com/components/espressif/esp32-camera

Could you please describe what behavior you are trying to achieve?

[xcguang]

@xcguang, the path field used together with git has a special meaning. It can be used to specify a directory inside a repo for the case when a component is not in the root of the repo.

If you want to use a component already existing on your machine, you can use a local directory instead.

dependencies:
  esp32-camera:
    path: "../../camera_components/"

Won't a component from the registry be a better option for you? https://components.espressif.com/components/espressif/esp32-camera

Could you please describe what behavior you are trying to achieve?

Yes, usually, we will use https://components.espressif.com/components/espressif/esp32-camera, but sometimes this component is behind the master branch, on which some bugs maybe have been fixed or some features are newly added, so to demo them, we are going to use https://github.com/espressif/esp32-camera.git in some examples of https://github.com/espressif/esp-iot-solution/tree/master/examples/camera.

[igrr]

but sometimes this component is behind the master branch, on which some bugs maybe have been fixed or some features are newly added

That sounds like the issue which should then be addressed. Releasing a new tag is really easy, so if you see that some feature you need is missing in a release, just mention this on the esp32-camera issue tracker.

[wujiangang]

BTW, for the git only component, can we specify the commit or tag or branch that the user wants to use? Currently, the default branch and the recent commit will be used.

[igrr]

You can do this via the version field:

dependencies:
  libnmea-esp32:
    git: https://github.com/igrr/libnmea-esp32.git
    version: feature/i2c

version can be anything Git recognizes: branch name, tag name, or a full commit hash.

[wujiangang]

Great, it would be useful in some cases. Is it documented somewhere? The version described on the wiki page seems not to include this part. Also no related statement on README.

[kumekay]

Great, it would be useful in some cases. Is it documented somewhere? The version described on the wiki page seems not to include this part. Also no related statement on README.

I'm working on the new documentation page about the component manager. This topic will be covered, it should be public next week

[kumekay]

Work with git repos is documented here: https://docs.espressif.com/projects/idf-component-manager/en/latest/reference/manifest_file.html#dependencies-from-git