Second execution of ./create.py -o fails because of conflicting ansible versions

[moschlar]

$ ./create.py -o jgu
[...]
+-----------------+
| pip install ... | (in .)
+-----------------+
| pip install ansible==2.7.5 --user
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: ansible==2.7.5 in /home/centos/.local/lib/python2.7/site-packages (2.7.5)
Requirement already satisfied: jinja2 in /usr/lib/python2.7/site-packages (from ansible==2.7.5) (2.7.2)
Requirement already satisfied: PyYAML in /usr/lib64/python2.7/site-packages (from ansible==2.7.5) (3.10)
Requirement already satisfied: paramiko in /home/centos/.local/lib/python2.7/site-packages (from ansible==2.7.5) (2.6.0)
Requirement already satisfied: cryptography in /home/centos/.local/lib/python2.7/site-packages (from ansible==2.7.5) (2.7)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from ansible==2.7.5) (41.0.1)
Requirement already satisfied: markupsafe in /usr/lib64/python2.7/site-packages (from jinja2->ansible==2.7.5) (0.11)
Requirement already satisfied: pynacl>=1.0.1 in /home/centos/.local/lib/python2.7/site-packages (from paramiko->ansible==2.7.5) (1.3.0)
Requirement already satisfied: bcrypt>=3.1.3 in /home/centos/.local/lib/python2.7/site-packages (from paramiko->ansible==2.7.5) (3.1.7)
Requirement already satisfied: enum34; python_version < "3" in /home/centos/.local/lib/python2.7/site-packages (from cryptography->ansible==2.7.5) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /home/centos/.local/lib/python2.7/site-packages (from cryptography->ansible==2.7.5) (0.24.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/centos/.local/lib/python2.7/site-packages (from cryptography->ansible==2.7.5) (1.12.3)
Requirement already satisfied: six>=1.4.1 in /home/centos/.local/lib/python2.7/site-packages (from cryptography->ansible==2.7.5) (1.12.0)
Requirement already satisfied: ipaddress; python_version < "3" in /home/centos/.local/lib/python2.7/site-packages (from cryptography->ansible==2.7.5) (1.0.22)
Requirement already satisfied: pycparser in /home/centos/.local/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography->ansible==2.7.5) (2.19)

+------------------+
| git checkout ... | (in openshift-ansible)
+------------------+
| git checkout tags/openshift-ansible-3.11.97-1
HEAD is now at 5bb60b0... Automatic commit of package [openshift-ansible] release [3.11.97-1].

+--------------------------------+
| ansible-playbook site.yaml ... | (in ansible/pre-okd)
+--------------------------------+
| ansible-playbook site.yaml  -i ../../okd/inventories/standard-3-11/inventory.yaml
Traceback (most recent call last):
  File "/usr/bin/ansible-playbook", line 32, in <module>
    from ansible import context
ImportError: cannot import name context
ERROR: Process failed (returncode=1)
# stderr follows...
ERROR: Failed to create

If I remove the Ansible rpm from the system, it works again for the next run.

[alanbchristie]

In the early revisions of the orchestrator I uninstalled the pre-existing version of Ansible but reverted this because I felt it too destructive.

Was it a system 2.7.5 that was installed that you removed?

[moschlar]

Was it a system 2.7.5 that was installed that you removed?

No, its 0:2.8.2-1.el7...

[alanbchristie]

The fix for this might be to introduce sudo to the orchestrator. Replacing the installed ansible (by not using --user) seems to fix the problem. Maybe we need to amend the pip adjustment section in create.py (around line 128) with something like...

    cmd = 'sudo pip install --upgrade pip'
    rv, _ = io.run(cmd, '.', cli_args.quiet)
    if not rv:
        return False

    cmd = 'sudo pip install --upgrade pip setuptools'
    rv, _ = io.run(cmd, '.', cli_args.quiet)
    if not rv:
        return False

    cmd = 'sudo pip install ansible=={}'. \
        format(deployment.okd.ansible_version)
    rv, _ = io.run(cmd, '.', cli_args.quiet)
    if not rv:
        return False

[moschlar]

Hm, but you also install a version of Ansible in the image building process (from yacker/files/okd-requirements.txt).

[moschlar]

And isn't it generally a bad idea to mix such packages from the distribution and another package management mechanism? :grimacing:

[alanbchristie]

Yep, mixing is bad, agreed and if there's a basic fault we should fix it.

More often though it turns out to be a battle with Python modules that have been installed in distributions with package managers other than PIP.

I have found the underlying yacker-built machine images to be generally suitable for the existing array of deployments. I have tended to re-build the infrastructure (machine instances) between installations rather than "try again" on potentially "dirty" machines though.

Thanks for persevering with this. It is all helpful.

[moschlar]

No problem, this kind of experimentation is where most new stuff is learned :grin:

[alanbchristie]

I've re-opened this because using sudo breaks the in-docker execution of the orchestrator.

I'm beginning to feel that it's potentially very complicated for the orchestrator to be able to cope with any host configuration/setup and that it's the host that needs adjusting, not the orchestrator. Adding a command-line option to enable sudo (so it runs without sudo in the container and with sudo outside) just feels messy.

The sudo changes (in master) have been undone and we should see what needs to be done on a "clean" example of the problem machine.