search

aristanetworks / avd

Arista Validated Designs
https://avd.arista.com
Apache License 2.0
295 stars 211 forks source link

Could not generate config with arista avd version v4.9.0 #4213

Closed duddis1 closed 2 months ago

duddis1 commented 3 months ago

Issue Summary

We are trying to adopt the new AVD version changes (v4.9.0), but we are facing the following issues while generating the config.

It seems the new AVD version requires Ansible version 8.0.0. We have updated Ansible to this version, along with the dependency on pyavd.

While installing the dependencies listed below, we are encountering the following error: pipenv install -r /home/avd/.ansible/collections/ansible_collections/arista/avd/requirements.txt

[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: metadata generation failed

We tried to use the command below. This time it did not give any errors, but the config was not generated for the devices/switches. /home/avd/.local/bin/pipenv install "pyavd[ansible-collection]==4.9.0" --skip-lock

Can you please provide us with the correct pipenv version we need to use, or any other library versions we need to change?

Which component(s) of AVD impacted

arista.avd.default

How do you run AVD ?

None

Steps to reproduce

Run below two commands inside pipenv along with other libraries.

pipenv install -r /home/avd/.ansible/collections/ansible_collections/arista/avd/requirements.txt

Relevant log output

[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: metadata generation failed

Contributing Guide

gmuloc commented 3 months ago

Hi @duddis1 thanks for reaching out here. Not many users of AVD (that we know of) are using pipenv so we will need to take a deeper look to understand what could. be the issue.

You say you have run:

/home/avd/.local/bin/pipenv install "pyavd[ansible-collection]==4.9.0" --skip-lock

following the error message. Can you please share the pipenv graph output?

Thanks

duddis1 commented 3 months ago

HI @gmuloc Please find below pipenv graph output after running the above command inside our script.

ansible==8.0.0
  - ansible-core [required: ~=2.15.0, installed: 2.15.12]
    - cryptography [required: Any, installed: 42.0.8]
      - cffi [required: >=1.12, installed: 1.16.0]
        - pycparser [required: Any, installed: 2.22]
    - importlib-resources [required: >=5.0,<5.1, installed: 5.0.7]
    - jinja2 [required: >=3.0.0, installed: 3.1.4]
      - MarkupSafe [required: >=2.0, installed: 2.1.5]
    - packaging [required: Any, installed: 24.1]
    - PyYAML [required: >=5.1, installed: 6.0.1]
    - resolvelib [required: >=0.5.3,<1.1.0, installed: 1.0.1]
anta==1.0.0
  - aiocache [required: >=0.12.2, installed: 0.12.2]
  - asyncssh [required: >=2.13.2, installed: 2.15.0]
    - cryptography [required: >=39.0, installed: 42.0.8]
      - cffi [required: >=1.12, installed: 1.16.0]
        - pycparser [required: Any, installed: 2.22]
    - typing-extensions [required: >=4.0.0, installed: 4.12.2]
  - cvprac [required: >=1.3.1, installed: 1.4.0]
    - packaging [required: >=23.2, installed: 24.1]
    - requests [required: >=2.27.0, installed: 2.28.1]
      - certifi [required: >=2017.4.17, installed: 2024.7.4]
      - charset-normalizer [required: >=2,<3, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.7]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
  - eval-type-backport [required: >=0.1.3, installed: 0.2.0]
  - httpx [required: >=0.27.0, installed: 0.27.0]
    - anyio [required: Any, installed: 4.4.0]
      - exceptiongroup [required: >=1.0.2, installed: 1.2.2]
      - idna [required: >=2.8, installed: 3.7]
      - sniffio [required: >=1.1, installed: 1.3.1]
      - typing-extensions [required: >=4.1, installed: 4.12.2]
    - certifi [required: Any, installed: 2024.7.4]
    - httpcore [required: ==1.*, installed: 1.0.5]
      - certifi [required: Any, installed: 2024.7.4]
      - h11 [required: >=0.13,<0.15, installed: 0.14.0]
    - idna [required: Any, installed: 3.7]
    - sniffio [required: Any, installed: 1.3.1]
  - Jinja2 [required: >=3.1.2, installed: 3.1.4]
    - MarkupSafe [required: >=2.0, installed: 2.1.5]
  - pydantic [required: >=2.7, installed: 2.8.2]
    - annotated-types [required: >=0.4.0, installed: 0.7.0]
    - pydantic-core [required: ==2.20.1, installed: 2.20.1]
      - typing-extensions [required: >=4.6.0,!=4.7.0, installed: 4.12.2]
    - typing-extensions [required: >=4.6.1, installed: 4.12.2]
  - pydantic-extra-types [required: >=2.3.0, installed: 2.9.0]
    - pydantic [required: >=2.5.2, installed: 2.8.2]
      - annotated-types [required: >=0.4.0, installed: 0.7.0]
      - pydantic-core [required: ==2.20.1, installed: 2.20.1]
        - typing-extensions [required: >=4.6.0,!=4.7.0, installed: 4.12.2]
      - typing-extensions [required: >=4.6.1, installed: 4.12.2]
  - PyYAML [required: >=6.0, installed: 6.0.1]
  - requests [required: >=2.31.0, installed: 2.28.1]
    - certifi [required: >=2017.4.17, installed: 2024.7.4]
    - charset-normalizer [required: >=2,<3, installed: 2.1.1]
    - idna [required: >=2.5,<4, installed: 3.7]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
  - rich [required: >=13.5.2,<14, installed: 13.7.1]
    - markdown-it-py [required: >=2.2.0, installed: 3.0.0]
      - mdurl [required: ~=0.1, installed: 0.1.2]
    - pygments [required: >=2.13.0,<3.0.0, installed: 2.18.0]
errorhandler==2.0.1
hvac-utils==1.1.7
  - hvac [required: ==0.11.2, installed: 0.11.2]
    - requests [required: >=2.21.0, installed: 2.28.1]
      - certifi [required: >=2017.4.17, installed: 2024.7.4]
      - charset-normalizer [required: >=2,<3, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.7]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
    - six [required: >=1.5.0, installed: 1.16.0]
  - requests [required: ==2.28.1, installed: 2.28.1]
    - certifi [required: >=2017.4.17, installed: 2024.7.4]
    - charset-normalizer [required: >=2,<3, installed: 2.1.1]
    - idna [required: >=2.5,<4, installed: 3.7]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
logextensions==2.1.6
  - termcolor [required: Any, installed: 2.4.0]
md-toc==8.2.3
  - fpyutils [required: >=4.0.1,<5, installed: 4.0.1]
netaddr==1.3.0
netbox-utils==1.1.11
  - pynetbox [required: ==6.6.2, installed: 6.6.2]
    - requests [required: >=2.20.0,<3.0, installed: 2.28.1]
      - certifi [required: >=2017.4.17, installed: 2024.7.4]
      - charset-normalizer [required: >=2,<3, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.7]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
    - six [required: ==1.*, installed: 1.16.0]
  - requests [required: ==2.28.1, installed: 2.28.1]
    - certifi [required: >=2017.4.17, installed: 2024.7.4]
    - charset-normalizer [required: >=2,<3, installed: 2.1.1]
    - idna [required: >=2.5,<4, installed: 3.7]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
  - semver [required: >=2.13,<3.0, installed: 2.13.0]
paramiko==3.4.0
  - bcrypt [required: >=3.2, installed: 4.1.3]
  - cryptography [required: >=3.3, installed: 42.0.8]
    - cffi [required: >=1.12, installed: 1.16.0]
      - pycparser [required: Any, installed: 2.22]
  - pynacl [required: >=1.5, installed: 1.5.0]
    - cffi [required: >=1.4.1, installed: 1.16.0]
      - pycparser [required: Any, installed: 2.22]
pyavd==4.9.0
  - aristaproto [required: >=0.1.1, installed: 0.1.2]
    - grpclib [required: >=0.4.1,<0.5.0, installed: 0.4.7]
      - h2 [required: >=3.1.0,<5, installed: 4.1.0]
        - hpack [required: >=4.0,<5, installed: 4.0.0]
        - hyperframe [required: >=6.0,<7, installed: 6.0.1]
      - multidict [required: Any, installed: 6.0.5]
    - python-dateutil [required: >=2.8,<3.0, installed: 2.9.0.post0]
      - six [required: >=1.5, installed: 1.16.0]
    - typing-extensions [required: >=4.7.1,<5.0.0, installed: 4.12.2]
  - cryptography [required: >=38.0.4, installed: 42.0.8]
    - cffi [required: >=1.12, installed: 1.16.0]
      - pycparser [required: Any, installed: 2.22]
  - deepmerge [required: >=1.1.0, installed: 1.1.1]
  - Jinja2 [required: >=3.0, installed: 3.1.4]
    - MarkupSafe [required: >=2.0, installed: 2.1.5]
  - jsonschema [required: >=4.10.3, installed: 4.23.0]
    - attrs [required: >=22.2.0, installed: 23.2.0]
    - jsonschema-specifications [required: >=2023.03.6, installed: 2023.12.1]
      - referencing [required: >=0.31.0, installed: 0.35.1]
        - attrs [required: >=22.2.0, installed: 23.2.0]
        - rpds-py [required: >=0.7.0, installed: 0.19.0]
    - referencing [required: >=0.28.4, installed: 0.35.1]
      - attrs [required: >=22.2.0, installed: 23.2.0]
      - rpds-py [required: >=0.7.0, installed: 0.19.0]
    - rpds-py [required: >=0.7.1, installed: 0.19.0]
  - requests [required: >=2.27.0, installed: 2.28.1]
    - certifi [required: >=2017.4.17, installed: 2024.7.4]
    - charset-normalizer [required: >=2,<3, installed: 2.1.1]
    - idna [required: >=2.5,<4, installed: 3.7]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.19]
PySocks==1.7.1
pytz==2024.1
ruamel.yaml==0.18.6
  - ruamel.yaml.clib [required: >=0.2.7, installed: 0.2.8]
treelib==1.7.0
  - six [required: Any, installed: 1.16.0]
gmuloc commented 3 months ago

thanks for sharing

Looking into this - for now I am not hitting the same issue as you.

Quick comments:

It seems the new AVD version requires Ansible version 8.0.0.

AVD does not require ansible, only ansible-core>=2.15.0,<2.18.0. can you please share how you reached this conclusion?

Also which python version are you using?

duddis1 commented 3 months ago

ansible-core [required: ~=2.15.0, installed: 2.15.12] is upgraded when I install ansible version 8.0.0 We are using python version 3.9

gmuloc commented 3 months ago

thanks for confirming - will find an env with 3.9 to see if I can repro with pipenv. Please notice that python 3.9 support will be removed in AVD 5.0 as we will drop support for Ansible 2.15 which is the lateste version supporting python 3.9: https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#ansible-core-support-matrix

So indeed using python 3.9, you are stuck with ansible-core 2.15.

Regarding ansible python package - you technically don't need it to run AVD - only ansible-core. Are you installing it for another reason?

gmuloc commented 3 months ago

So far not able to repro even using 3.9

can you recreate the install issue or are you now stuck in the situation where you cannot generate the configs ?

If you can recreate could you rerun the failing install using ´pipenv install --verbose’  ?

For the config not being generated, could you share the playbook verbose logs (-vvv) via email with me: gmulocher@arista.com ?

duddis1 commented 3 months ago

I could able to generate the config in different environment, but not sure the reason behind generating the config in current environment, as there was no error.

Below is the verbose log while installing: /home/avd/.local/bin/pipenv install "pyavd[ansible-collection]==4.9.0" -

INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.backtracking(LinkCandidate('https://artifactory.ros.gov.uk/artifactory/api/pypi/ros-pypi-remote/packages/packages/76/8e/80a74a719a503420abe3151a52f6053eb10447 f240348d20245c8626d504/errorhandler-2.0.0-py3-none-any.whl#sha256=1138c583436886d597cca41648229707abc60f42d405ee641939d039401a2a2f (from https://artifactory.ros.gov.uk/artifactory/api/pypi/ros-pypi-remote/simple/errorhandler/)'))

ERROR:pip.subprocessor: python setup.py egg_info exited with 1 Traceback (most recent call last): File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata call_subprocess( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/utils/subprocess.py", line 224, in call_subprocess raise error pipenv.patched.pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/avd/.local/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 685, in resolve results = resolver.resolve(self.constraints, check_supported_wheels=False) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve result = self._result = resolver.resolve( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 380, in resolve success = self._backtrack() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 320, in _backtrack success = _patch_criteria() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 309, in _patch_criteria if not candidates: File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_vendor/resolvelib/structs.py", line 151, in bool return bool(self._sequence) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in bool return any(self) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in return (c for c in iterator if id(c) not in self._incompatible_ids) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built candidate = func() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link self._link_candidate_cache = LinkCandidate( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 301, in init super().init( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 163, in init self.dist = self._prepare() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 232, in _prepare dist = self._prepare_distribution() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 312, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement dist = _get_prepared_distribution( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution abstract_dist.prepare_distribution_metadata( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata self.req.prepare_metadata() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/req/req_install.py", line 546, in prepare_metadata self.metadata_directory = generate_metadata_legacy( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/patched/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata raise MetadataGenerationFailed(package_details=details) from error pipenv.patched.pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/avd/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 845, in main() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 831, in main _main( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 811, in _main resolve_packages( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 759, in resolve_packages results, resolver = resolve( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 738, in resolve return resolve_deps( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 1100, in resolve_deps results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps( File "/home/avd/.local/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 899, in actually_resolve_deps resolver.resolve() File "/home/avd/.local/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 687, in resolve raise ResolutionFailure(message=str(e)) pipenv.exceptions.ResolutionFailure: [1m[31m[1m[1mERROR[1m[0m: [1m[33mmetadata generation failed[1m[0m Locking Failed! Locking...

===================================================================================

Can you please advise which python version we have to be upgraded to to adopt arista avd version v5.x

gmuloc commented 3 months ago

Hello - no progress for now on the pipenv front but wanted to

  1. thank you for sharing the verbose log
  2. ask you if you have by any chance any .env file in your pipenv folder (and if so could you please see what happens when you remove it)
  3. answer your question - AVD 5.x will support python 3.10-3.12 when released and we will be adding 3.13 when it makes sense. I would recommend moving to at the minimum python 3.11. But if you can to python 3.12.
duddis1 commented 3 months ago

"I found the reason for the configuration not being generated. I had placed the grpc_tunnels in the wrong location inside the JSON structure. However, it did not give me any error, and the configuration was not generated.

After removing it from the json, the configuration was successfully generated for version 4.9.0. The issue is now resolved."

            "management_api_gnmi": {
                "grpc_tunnels": {
                    "shutdown": true
                },
                "transport": {
                    "grpc": [
                        {
                            "name": "default"
                        }
                    ]
                }
            },
        "management_api_gnmi": {
            "transport": {
                "grpc": [
                    {
                        "name": "default"
                    }
                ]
            }
        },
gmuloc commented 3 months ago

ok so it seems that you are not getting the validation error and a silent treatment which is not good news. let me investigate further! thanks for coming back

ClausHolbechArista commented 3 months ago

@duddis1 please confirm if all issues mentioned here have been resolved. Thanks!

duddis1 commented 3 months ago

yes, these issue has been resolved, when you are going to release Arista 5.x version?

ClausHolbechArista commented 2 months ago

4.10 has been released including this fix. 5.0 is being worked on actively. We expect it to be released around October.

Please confirm that we can close this issue. Thanks.