Python six/jinja2 cookiecutter dependency version conflict causing CI builds to fail

[geerlingguy]

Failed build: https://travis-ci.org/github/geerlingguy/ansible-role-firewall/jobs/677476167

$ molecule test
Failed to load driver entry point Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 267, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2410, in load
    self.require(*args, **kwargs)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2433, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (six 1.11.0 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('six>=1.14'), {'cookiecutter'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/molecule/api.py", line 44, in drivers
    pm.load_setuptools_entrypoints("molecule.driver")
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 273, in load_setuptools_entrypoints
    message="Plugin %r could not be loaded: %s!" % (ep.name, e),
pluggy.manager.PluginValidationError: Plugin 'delegated' could not be loaded: (six 1.11.0 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('six>=1.14'), {'cookiecutter'})!

Failed to load verifier entry point Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 267, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2410, in load
    self.require(*args, **kwargs)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2433, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (six 1.11.0 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('six>=1.14'), {'cookiecutter'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/molecule/api.py", line 65, in verifiers
    pm.load_setuptools_entrypoints("molecule.verifier")
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 273, in load_setuptools_entrypoints
    message="Plugin %r could not be loaded: %s!" % (ep.name, e),
pluggy.manager.PluginValidationError: Plugin 'ansible' could not be loaded: (six 1.11.0 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('six>=1.14'), {'cookiecutter'})!

---
dependency:
  name: galaxy
driver:
  name: docker
lint: 'set -e

  yamllint .

  ansible-lint

  '
platforms:
  - command: ${MOLECULE_DOCKER_COMMAND:-""}
    image: geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:latest
    name: instance
    pre_build_image: true
    privileged: true
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:ro
provisioner:
  name: ansible
  playbooks:
    converge: ${MOLECULE_PLAYBOOK:-converge.yml}
scenario:
  test_sequence:
    - lint
    - destroy
    - dependency
    - syntax
    - create
    - prepare
    - converge
    - idempotence
    - check
    - side_effect
    - verify
    - destroy

ERROR: Failed to pre-validate.

{'driver': [{'name': ['unallowed value docker']}]}
The command "molecule test" exited with 1.

[geerlingguy]

Looking at the pip install molecule yamllint ansible-lint docker earlier in the setup, I see:

cookiecutter 1.7.1 has requirement Jinja2<=2.11.0, but you'll have jinja2 2.11.2 which is incompatible.
cookiecutter 1.7.1 has requirement six>=1.14, but you'll have six 1.11.0 which is incompatible.

[geerlingguy]

Now complaining about Jinja2:

raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (Jinja2 2.11.2 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('Jinja2<=2.11.0'), {'cookiecutter'})

[geerlingguy]

Might be related to cookiecutter's 1.7.1 release, which could spell doom for the rest of my CI jobs, since it seems cookiecutter is used by one of the things I install everywhere :(

https://github.com/cookiecutter/cookiecutter/commit/b943abfd50b005da06cf68314871019a6ec184c7

[geerlingguy]

Related, possibly: https://github.com/cookiecutter/cookiecutter/issues/1256#issuecomment-616855417

[geerlingguy]

Cookiecutter dependency in molecule is constrained to cookiecutter >= 1.6.0 (see https://github.com/ansible-community/molecule/blob/c65488c4acb61fe2628b7f32e70b13998957a131/setup.cfg#L77).

Attempting to lock in 1.7.0 with pip install cookiecutter==1.7.0 prior to installing molecule...

[geerlingguy]

That worked. So fix for now is to add before pip install molecule ansible-lint, pip install cookiecutter==1.7.0. Hopefully the dep resolution will be able to be fixed in cookiecutter though, or molecule can set it to be 'up to but not including 1.7.1'

[geerlingguy]

Looks like this was fixed upstream in the cookiecutter project, woohoo! 1.7.2 release fixes the too-strict dependencies.