search

ansible / mazer

Experimental Ansible Galaxy Content Manager
GNU General Public License v3.0
114 stars 18 forks source link

Conflicting Dependencies are not handled or reported to the user #255

Closed ironfroggy closed 5 years ago

ironfroggy commented 5 years ago

Bug Report

MAZER VERSION
name = mazer
version = 0.5.0
config_file = /home/calvin/.ansible/mazer.yml
uname = Linux, willow-epsilon, 4.4.0-17134-Microsoft, #706-Microsoft Mon Apr 01 18:13:00 PST 2019, x86_64
executable_location = /mnt/c/Users/calvin/redhat/.virtualenvs/orion/bin/mazer
python_version = 3.7.1 (default, Oct 22 2018, 11:21:55) [GCC 8.2.0]
python_executable = /mnt/c/Users/calvin/redhat/.virtualenvs/orion/bin/python3.7
CONFIGURATION

Default configuration.

SUMMARY

If installing a collection which leads to a conflict in dependencies, by way of two collections it depends on having their own dependencies on a forth collection, but different versions of it, hte installation blows up without an indication of the problem.

STEPS TO REPRODUCE
EXPECTED RESULTS

I did not expect this to work, but I was testing to find out how it was handled. Primarily, I wanted to confirm it doesn't silently overwrite one version of

ACTUAL RESULTS

The first three collections are installed, but the force causes an error. The installed collections are now not usable, and the user does not know why the installation failed.

Traceback (most recent call last):
  File "/mnt/c/Users/calvin/redhat/mazer/ansible_galaxy/actions/install.py", line 211, in find_new_deps_from_installed
    deps_and_reqs_list = sorted(list(deps_and_reqs_set))
  File "/mnt/c/Users/calvin/redhat/.virtualenvs/orion/lib/python3.7/site-packages/attr/_make.py", line 1148, in __lt__
    return attrs_to_tuple(self) < attrs_to_tuple(other)
  File "/mnt/c/Users/calvin/redhat/.virtualenvs/orion/lib/python3.7/site-packages/attr/_make.py", line 1148, in __lt__
    return attrs_to_tuple(self) < attrs_to_tuple(other)
TypeError: '<' not supported between instances of 'Spec' and 'Spec'
alikins commented 5 years ago

@ironfroggy Do you have example collections available or published somewhere I could use to reproduce?

ironfroggy commented 5 years ago

My test automates a bunch of temporary collections when it runs... Give me a bit and I'll extract to create a permanent example.

ironfroggy commented 5 years ago

"Collection A" https://galaxy-qa.ansible.com/collections/orionuser1/collection_dep_a_bvsivsmc depends on orionuser1.collection_dep_a_dpvsiwdd==1.0.0, and orionuser1.collection_dep_a_kmcnjmib==1.0.0

"Collection B" https://galaxy-qa.ansible.com/collections/orionuser1/collection_dep_a_dpvsiwdd depends on orionuser1.collection_dep_a_vstghmmb==2.0.0

"Collection C" https://galaxy-qa.ansible.com/collections/orionuser1/collection_dep_a_kmcnjmib depends on orionuser1.collection_dep_a_vstghmmb==1.0.0

"Collection D" https://galaxy-qa.ansible.com/collections/orionuser1/collection_dep_a_vstghmmb has no dependencies, but is depended on

@alikins If you install the first one, you'll have A, B, and C locally to inspect. You can install the last one directly then to inspect it, but there's really nothing to that one.

alikins commented 5 years ago

Guessing that the not entirely necessary sorted() in there is causing the list of Requirement() objects to get compared, and when it's comparing the attributes and hits the RequirementSpec.version_spec which is a semantic_version.Spec() and not comparable (the '< is not supported')

Will probably just remove the sorted() since it's not required, but was there to force an order after losing ordering after using the set() to uniq'ify the list. Guess it needs an order preserving uniq there. (Trying to make sure that the list of requirements to install has some predictable ordering so that dep solving happens in the same order each time to make it easier to troubleshoot).

alikins commented 5 years ago

Another possibility is that the Requirement object needs to make sure 'version_spec' isn't used for cmp.

alikins commented 5 years ago

I can reproduce with devel as mentioned above:

(mazer040test) [newswoop:F29:collection_inspect (master % u=)]$ mazer install --collections-path=/tmp/coltest2 --server http://galaxy-qa.ansible.com orionuser1.collection_dep_a_bvsivsmc
Installing orionuser1.collection_dep_a_bvsivsmc (version_spec: *)
- The repository orionuser1.collection_dep_a_bvsivsmc was successfully installed to /tmp/coltest2
Installing requirement orionuser1.collection_dep_a_dpvsiwdd (version_spec: ==1.0.0) (required by orionuser1.collection_dep_a_bvsivsmc)
Installing requirement orionuser1.collection_dep_a_kmcnjmib (version_spec: ==1.0.0) (required by orionuser1.collection_dep_a_bvsivsmc)
- The repository orionuser1.collection_dep_a_dpvsiwdd was successfully installed to /tmp/coltest2
- The repository orionuser1.collection_dep_a_kmcnjmib was successfully installed to /tmp/coltest2
'<' not supported between instances of 'Spec' and 'Spec'
Traceback (most recent call last):
  File "/home/adrian/venvs/mazer040test/bin/mazer", line 11, in <module>
    load_entry_point('mazer', 'console_scripts', 'mazer')()
  File "/home/adrian/src/mazer/ansible_galaxy_cli/main.py", line 36, in main
    exit_code = cli.run()
  File "/home/adrian/src/mazer/ansible_galaxy_cli/cli/galaxy.py", line 217, in run
    return self.execute()
  File "/home/adrian/src/mazer/ansible_galaxy_cli/cli/__init__.py", line 152, in execute
    return fn()
  File "/home/adrian/src/mazer/ansible_galaxy_cli/cli/galaxy.py", line 283, in execute_install
    force_overwrite=self.options.force)
  File "/home/adrian/src/mazer/ansible_galaxy/actions/install.py", line 172, in install_repository_specs_loop
    no_deps=no_deps)
  File "/home/adrian/src/mazer/ansible_galaxy/actions/install.py", line 210, in find_new_deps_from_installed
    deps_and_reqs_list = sorted(list(deps_and_reqs_set))
  File "/home/adrian/venvs/mazer040test/lib/python3.6/site-packages/attr/_make.py", line 1148, in __lt__
    return attrs_to_tuple(self) < attrs_to_tuple(other)
  File "/home/adrian/venvs/mazer040test/lib/python3.6/site-packages/attr/_make.py", line 1148, in __lt__
    return attrs_to_tuple(self) < attrs_to_tuple(other)
TypeError: '<' not supported between instances of 'Spec' and 'Spec'

260 has a potential fix for the TypeError

With #260 applied

(mazer040test) [newswoop:F29:collection_inspect (master % u=)]$ rm -rf /tmp/coltest2/* && mazer install --collections-path=/tmp/coltest2 --server http://galaxy-qa.ansible.com orionuser1.collection_dep_a_bvsivsmc
Installing orionuser1.collection_dep_a_bvsivsmc (version_spec: *)
- The repository orionuser1.collection_dep_a_bvsivsmc was successfully installed to /tmp/coltest2
Installing requirement orionuser1.collection_dep_a_dpvsiwdd (version_spec: ==1.0.0) (required by orionuser1.collection_dep_a_bvsivsmc)
Installing requirement orionuser1.collection_dep_a_kmcnjmib (version_spec: ==1.0.0) (required by orionuser1.collection_dep_a_bvsivsmc)
- The repository orionuser1.collection_dep_a_dpvsiwdd was successfully installed to /tmp/coltest2
- The repository orionuser1.collection_dep_a_kmcnjmib was successfully installed to /tmp/coltest2
Installing requirement orionuser1.collection_dep_a_vstghmmb (version_spec: ==2.0.0) (required by orionuser1.collection_dep_a_dpvsiwdd)
Installing requirement orionuser1.collection_dep_a_vstghmmb (version_spec: ==1.0.0) (required by orionuser1.collection_dep_a_kmcnjmib)
- The repository orionuser1.collection_dep_a_vstghmmb was successfully installed to /tmp/coltest2
WARNING| - orionuser1.collection_dep_a_vstghmmb was NOT installed successfully: The Galaxy content /tmp/coltest2/ansible_collections/orionuser1/collection_dep_a_vstghmmb/README.md appears to already exist. 
The Galaxy content /tmp/coltest2/ansible_collections/orionuser1/collection_dep_a_vstghmmb/README.md appears to already exist.:
- you can use --ignore-errors to skip failed collections and finish processing the list.

Still kind of a confusing error though. Can improve that slightly by including version info in some of the messages that just mention the collection id now.

Showing much more useful detail that that will likely have to wait until the dep solver is rewritten to be 'real' dep solver.

alikins commented 5 years ago

See #261 for hopefully a slightly better output as mentioned inhttps://github.com/ansible/mazer/issues/255#issuecomment-493484408