search

ansible / molecule

Molecule aids in the development and testing of Ansible content: collections, playbooks and roles
https://ansible.readthedocs.io/projects/molecule/
MIT License
3.84k stars 659 forks source link

Default tests fail to run when remote_user is set to a different user than the current one #937

Closed coaxial closed 6 years ago

coaxial commented 6 years ago

Issue Type

Molecule and Ansible details

ansible --version
ansible 2.3.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170118]

molecule --version
molecule, version 2.0.1

Desired Behaviour

molecule test should run fine

Actual Behaviour (Bug report only)

$ molecule init role --role-name test-role
--> Initializing new role test-role...
Initialized role in /home/coaxial/code/test-role successfully.

$ cd test-role/

$ molecule test
--> Test matrix

└── default
    ├── destroy
    ├── dependency
    ├── syntax
    ├── create
    ├── converge
    ├── idempotence
    ├── lint
    ├── side_effect
    ├── verify
    └── destroy
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    ok: [localhost] => (item=(censored due to no_log))

    PLAY RECAP *********************************************************************
    localhost                  : ok=1    changed=0    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.
--> Scenario: 'default'
--> Action: 'syntax'

    playbook: /home/coaxial/code/test-role/molecule/default/playbook.yml

--> Scenario: 'default'
--> Action: 'create'

    PLAY [Create] ******************************************************************

    TASK [Create Dockerfiles from image names] *************************************
    changed: [localhost] => (item=(censored due to no_log))

    TASK [Discover local Docker images] ********************************************
    ok: [localhost] => (item=(censored due to no_log))

    TASK [Build an Ansible compatible image] ***************************************
    changed: [localhost] => (item=(censored due to no_log))

    TASK [Create molecule instance(s)] *********************************************
    changed: [localhost] => (item=(censored due to no_log))

    PLAY RECAP *********************************************************************
    localhost                  : ok=4    changed=3    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'converge'

    PLAY [Converge] ****************************************************************

    TASK [Gathering Facts] *********************************************************
    ok: [instance]

    PLAY RECAP *********************************************************************
    instance                   : ok=1    changed=0    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'idempotence'
Idempotence completed successfully.
--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /home/coaxial/code/test-role/...
Lint completed successfully.
--> Executing Flake8 on files found in /home/coaxial/code/test-role/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /home/coaxial/code/test-role/molecule/default/playbook.yml...
Lint completed successfully.
--> Scenario: 'default'
--> Action: 'side_effect'
Skipping, side effect playbook not configured.
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/coaxial/code/test-role/molecule/default/tests/...
    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.13, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
    rootdir: /home/coaxial/code/test-role/molecule/default, inifile:
    plugins: testinfra-1.6.3
collected 1 item                                                                

    tests/test_default.py F

    =================================== FAILURES ===================================
    _____________________ test_hosts_file[ansible://instance] ______________________

    host = <testinfra.host.Host object at 0x7f3ee7529d50>

        def test_hosts_file(host):
    >       f = host.file('/etc/hosts')

    tests/test_default.py:10:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    ../../../../.local/lib/python2.7/site-packages/testinfra/host.py:91: in __getattr__
        obj = module_class.get_module(self)
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/base.py:22: in get_module
        klass = cls.get_module_class(_host)
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/file.py:176: in get_module_class
        if host.system_info.type == "linux":
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/systeminfo.py:122: in type
        return self.sysinfo["type"]
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/systeminfo.py:31: in sysinfo
        self._sysinfo = self.get_system_info()
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/systeminfo.py:103: in get_system_info
        sysinfo["type"] = self.check_output("uname -s").lower()
    ../../../../.local/lib/python2.7/site-packages/testinfra/host.py:55: in run
        return self.backend.run(command, *args, **kwargs)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    self = <testinfra.backend.ansible.AnsibleBackend object at 0x7f3ee7529b90>
    command = 'uname -s', args = (), kwargs = {}, out = {}

        def run(self, command, *args, **kwargs):
            command = self.get_command(command, *args)
            out = self.run_ansible("shell", module_args=command)
            return self.result(
    >           out['rc'],
                command,
                stdout_bytes=None,
                stderr_bytes=None,
                stdout=out["stdout"], stderr=out["stderr"],
            )
    E       KeyError: u'rc'

    ../../../../.local/lib/python2.7/site-packages/testinfra/backend/ansible.py:47: KeyError
    ----------------------------- Captured stderr call -----------------------------
     [WARNING]: Failure using method (v2_runner_on_unreachable) in callback plugin
    (<testinfra.utils.ansible_runner.Callback object at 0x7f3ee7441790>): Host
    instance is unreachable: {'changed': False,  'msg': u'Authentication or
    permission failure. In some cases, you may have been able to authenticate and
    did not have permissions on the remote directory. Consider changing the remote
    temp path in ansible.cfg to a path rooted in "/tmp". Failed command was: (
    umask 77 && mkdir -p "` echo unable to find user ansible: no matching entries
    in passwd file/.ansible/tmp/ansible-tmp-1504303511.63-247028524748039 `" &&
    echo ansible-tmp-1504303511.63-247028524748039="` echo unable to find user
    ansible: no matching entries in passwd file/.ansible/tmp/ansible-
    tmp-1504303511.63-247028524748039 `" ), exited with result 126: unable to find
    user ansible: no matching entries in passwd file\r\n',  'unreachable': True}
    =============================== warnings summary ===============================
    None
      Module already imported so can not be re-written: testinfra

    -- Docs: http://doc.pytest.org/en/latest/warnings.html
    ===================== 1 failed, 1 warnings in 0.53 seconds =====================

$ molecule destroy
--> Test matrix

└── default
    └── destroy
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=(censored due to no_log))

    PLAY RECAP *********************************************************************
    localhost                  : ok=1    changed=1    unreachable=0    failed=0
$ pip freeze
adium-theme-ubuntu==0.3.4
ansible==2.3.2.0
ansible-lint==3.4.13
anyconfig==0.9.1
apsw==3.16.2.post1
arrow==0.10.0
asn1crypto==0.22.0
attrs==17.2.0
autobahn==17.6.2
Automat==0.6.0
backports.ssl-match-hostname==3.5.0.1
bcrypt==3.1.3
BeautifulSoup==3.2.1
beautifulsoup4==4.5.3
binaryornot==0.4.4
certifi==2017.7.27.1
cffi==1.10.0
chardet==3.0.4
Cheetah==2.4.4
CherryPy==3.5.0
click==6.7
click-completion==0.2.1
colorama==0.3.7
configparser==3.5.0
constantly==15.1.0
cookiecutter==1.5.1
cryptography==2.0.3
cssselect==1.0.1
cssutils==1.0
dblatex==0.3.9.post1
dnspython==1.15.0
docker-py==1.10.6
docker-pycreds==0.2.1
duplicity==0.7.6
enum34==1.1.6
fasteners==0.14.1
feedparser==5.1.3
flake8==3.3.0
future==0.16.0
git-url-parse==1.0.1
hkdf==0.0.3
howdoi==1.1.9
html5lib==0.999999999
httplib2==0.9.2
humanize==0.5.1
hyperlink==17.2.1
idna==2.6
incremental==17.5.0
ipaddr==2.1.11
ipaddress==1.0.18
Jinja2==2.9.6
jinja2-time==0.2.0
keyring==10.3.1
keyrings.alt==2.2
libvirt-python==3.0.0
lockfile==0.12.2
lxml==3.8.0
magic-wormhole==0.10.2
Markdown==2.6.8
MarkupSafe==1.0
marshmallow==2.13.5
mccabe==0.6.1
mechanize==0.2.5
mercurial==3.9.1
molecule==2.0.1
monotonic==1.3
ndg-httpsclient==0.4.2
netifaces==0.10.4
paramiko==2.2.1
pathspec==0.5.3
pbr==3.0.1
pexpect==4.2.1
Pillow==4.0.0
poyo==0.4.1
progressbar==2.3
psutil==5.2.2
ptyprocess==0.5.2
py==1.4.34
pyasn1==0.3.3
pyasn1-modules==0.0.9
pycodestyle==2.3.1
pycparser==2.18
pycrypto==2.6.1
pyflakes==1.5.0
Pygments==2.2.0
pygobject==3.22.0
PyNaCl==1.1.2
pyOpenSSL==17.0.0
pyparsing==2.1.10
pyquery==1.2.17
pytest==3.2.1
python-apt==1.4.0b2
python-dateutil==2.6.1
python-gilt==1.1.0
python-vagrant==0.5.14
pyxdg==0.25
PyYAML==3.12
repoze.lru==0.6
requests==2.18.4
requests-cache==0.4.13
Routes==2.2
SecretStorage==2.3.1
service-identity==17.0.0
sh==1.12.14
six==1.10.0
spake2==0.7
tabulate==0.7.7
testinfra==1.6.3
tqdm==4.14.0
tree-format==0.1.1
Twisted==17.5.0
txaio==2.8.0
txtorcon==0.19.3
unity-lens-photos==1.0
urllib3==1.22
uTidylib==0.3
webencodings==0.5
WebOb==1.6.2
websocket-client==0.44.0
whichcraft==0.4.1
yamllint==1.8.1
zope.interface==4.4.2
coaxial commented 6 years ago 
$ molecule --debug verify
--> Test matrix

└── default
    └── verify
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/coaxial/code/test-role/molecule/default/tests/...
DEBUG: ANSIBLE ENVIRONMENT
--- {}

DEBUG: MOLECULE ENVIRONMENT
---
MOLECULE_DEBUG: 'True'
MOLECULE_DEPENDENCY_NAME: galaxy
MOLECULE_DRIVER_NAME: docker
MOLECULE_EPHEMERAL_DIRECTORY: /home/coaxial/code/test-role/molecule/default/.molecule
MOLECULE_FILE: /home/coaxial/code/test-role/molecule/default/molecule.yml
MOLECULE_INSTANCE_CONFIG: /home/coaxial/code/test-role/molecule/default/.molecule/instance_config.yml
MOLECULE_INVENTORY_FILE: /home/coaxial/code/test-role/molecule/default/.molecule/ansible_inventory.yml
MOLECULE_LINT_NAME: yamllint
MOLECULE_PROVISIONER_NAME: ansible
MOLECULE_SCENARIO_DIRECTORY: /home/coaxial/code/test-role/molecule/default
MOLECULE_SCENARIO_NAME: default
MOLECULE_VERIFIER_NAME: testinfra

DEBUG: COMMAND
/home/coaxial/.local/bin/testinfra --debug --connection=ansible --ansible-inventory=/home/coaxial/code/test-role/molecule/default/.molecule/ansible_inventory.yml /home/coaxial/code/test-role/molecule/default/tests/test_default.py
writing pytestdebug information to /home/coaxial/code/test-role/molecule/default/pytestdebug.log
    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.13, pytest-3.2.1, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python
    using: pytest-3.2.1 pylib-1.4.34
    setuptools registered plugins:
      testinfra-1.6.3 at /home/coaxial/.local/lib/python2.7/site-packages/testinfra/plugin.py
    rootdir: /home/coaxial/code/test-role/molecule/default, inifile:
    plugins: testinfra-1.6.3
collected 1 item                                                                

    tests/test_default.py F

    =================================== FAILURES ===================================
    _____________________ test_hosts_file[ansible://instance] ______________________

    host = <testinfra.host.Host object at 0x7fe761cee1d0>

        def test_hosts_file(host):
    >       f = host.file('/etc/hosts')

    tests/test_default.py:10:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    ../../../../.local/lib/python2.7/site-packages/testinfra/host.py:91: in __getattr__
        obj = module_class.get_module(self)
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/base.py:22: in get_module
        klass = cls.get_module_class(_host)
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/file.py:176: in get_module_class
        if host.system_info.type == "linux":
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/systeminfo.py:122: in type
        return self.sysinfo["type"]
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/systeminfo.py:31: in sysinfo
        self._sysinfo = self.get_system_info()
    ../../../../.local/lib/python2.7/site-packages/testinfra/modules/systeminfo.py:103: in get_system_info
        sysinfo["type"] = self.check_output("uname -s").lower()
wrote pytestdebug information to /home/coaxial/code/test-role/molecule/default/pytestdebug.log
    ../../../../.local/lib/python2.7/site-packages/testinfra/host.py:55: in run
        return self.backend.run(command, *args, **kwargs)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    self = <testinfra.backend.ansible.AnsibleBackend object at 0x7fe761ceefd0>
    command = 'uname -s', args = (), kwargs = {}, out = {}

        def run(self, command, *args, **kwargs):
            command = self.get_command(command, *args)
            out = self.run_ansible("shell", module_args=command)
            return self.result(
    >           out['rc'],
                command,
                stdout_bytes=None,
                stderr_bytes=None,
                stdout=out["stdout"], stderr=out["stderr"],
            )
    E       KeyError: u'rc'

    ../../../../.local/lib/python2.7/site-packages/testinfra/backend/ansible.py:47: KeyError
    ----------------------------- Captured stderr call -----------------------------
     [WARNING]: Failure using method (v2_runner_on_unreachable) in callback plugin
    (<testinfra.utils.ansible_runner.Callback object at 0x7fe761c84b50>): Host
    instance is unreachable: {'changed': False,  'msg': u'Authentication or
    permission failure. In some cases, you may have been able to authenticate and
    did not have permissions on the remote directory. Consider changing the remote
    temp path in ansible.cfg to a path rooted in "/tmp". Failed command was: (
    umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-
    tmp-1504303852.35-12760609404146 `" && echo ansible-
    tmp-1504303852.35-12760609404146="` echo ~/.ansible/tmp/ansible-
    tmp-1504303852.35-12760609404146 `" ), exited with result 1',  'unreachable':
    True}
    =============================== warnings summary ===============================
    None
      Module already imported so can not be re-written: testinfra

    -- Docs: http://doc.pytest.org/en/latest/warnings.html
    ===================== 1 failed, 1 warnings in 0.53 seconds =====================
retr0h commented 6 years ago

@philpep seen such an error before?

retr0h commented 6 years ago

@coaxial I cannot reproduce.

I fired up a Vagrant 17.04 host OS and did the following.

root@tester:~/test-role# uname -a
Linux tester 4.10.0-33-generic #37-Ubuntu SMP Fri Aug 11 10:55:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@tester:~/test-role# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 17.04
Release:    17.04
Codename:   zesty
root@tester:~/test-role# apt-get update; apt-get upgrade -y
root@tester:~/test-role# sudo apt-get remove -y docker
root@tester:~/test-role# sudo apt-get install -y python-virtualenv python-pip libssl-dev docker.io
root@tester:~/test-role# sudo gpasswd -a ${USER} docker

root@tester:~/test-role#  virtualenv --no-site-packages venv
root@tester:~/test-role#  source venv/bin/activate
(venv) root@tester:~/test-role# pip install docker-py
(venv) root@tester:~/test-role# pip install molecule --pre
(venv) root@tester:~/test-role# molecule init role --role-name test-role
--> Initializing new role test-role...
Initialized role in /root/test-role/test-role successfully.
(venv) root@tester:~/test-role# cd test-role
(venv) root@tester:~/test-role# molecule test
--> Test matrix

└── default
    ├── destroy
    ├── dependency
    ├── syntax
    ├── create
    ├── converge
    ├── idempotence
    ├── lint
    ├── side_effect
    ├── verify
    └── destroy
--> Scenario: 'default'
--> Action: 'destroy'
...
Skipping, side effect playbook not configured.
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /root/test-role/molecule/default/tests/...
    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.13, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
    rootdir: /root/test-role/molecule/default, inifile:
    plugins: testinfra-1.6.3
collected 1 item

    tests/test_default.py .

    =============================== warnings summary ===============================
    None
      Module already imported so can not be re-written: testinfra

    -- Docs: http://doc.pytest.org/en/latest/warnings.html
    ===================== 1 passed, 1 warnings in 3.88 seconds =====================
Verifier completed successfully.
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=(censored due to no_log))

    PLAY RECAP *********************************************************************
    localhost                  : ok=1    changed=1    unreachable=0    failed=0
root@tester:~/test-role# pip freeze
ansible==2.3.2.0
ansible-lint==3.4.13
anyconfig==0.9.1
arrow==0.10.0
asn1crypto==0.22.0
backports.ssl-match-hostname==3.5.0.1
bcrypt==3.1.3
binaryornot==0.4.4
certifi==2017.7.27.1
cffi==1.10.0
chardet==3.0.4
click==6.7
click-completion==0.2.1
colorama==0.3.7
configparser==3.5.0
cookiecutter==1.5.1
cryptography==2.0.3
docker-py==1.10.6
docker-pycreds==0.2.1
enum34==1.1.6
fasteners==0.14.1
flake8==3.3.0
future==0.16.0
git-url-parse==1.0.1
idna==2.6
ipaddress==1.0.18
Jinja2==2.9.6
jinja2-time==0.2.0
MarkupSafe==1.0
marshmallow==2.13.5
mccabe==0.6.1
molecule==2.0.1
monotonic==1.3
paramiko==2.2.1
pathspec==0.5.3
pbr==3.0.1
pexpect==4.2.1
pkg-resources==0.0.0
poyo==0.4.1
psutil==5.2.2
ptyprocess==0.5.2
py==1.4.34
pyasn1==0.3.3
pycodestyle==2.3.1
pycparser==2.18
pycrypto==2.6.1
pyflakes==1.5.0
PyNaCl==1.1.2
pytest==3.2.1
python-dateutil==2.6.1
python-gilt==1.1.0
PyYAML==3.12
requests==2.18.4
sh==1.12.14
six==1.10.0
tabulate==0.7.7
testinfra==1.6.3
tree-format==0.1.1
urllib3==1.22
websocket-client==0.44.0
whichcraft==0.4.1
yamllint==1.8.1
coaxial commented 6 years ago

Maybe we're using a different docker version?

$ docker --version
Docker version 17.06.1-ce, build 874a737
coaxial commented 6 years ago

I think I've found the cause.

In my /etc/ansible/ansible.cfg, I have remote_user = ansible. Which is different from the default ansible user - $(whoami).

If I remove that line and go with the current user, then the tests pass. So I'm guessing molecule or testinfra assumes the default user will connect and doesn't check if remote_user is different. It fails to copy the public key for remote_user, and as a result it can't log in to the container.