Molecule verify fails due to pytest collection errors

[0xdeadbeefJERKY]

🐛 Summary

During molecule "verify" phase, pytest errors out when attempting to collect the test cases.

To reproduce

Steps to reproduce the behavior:

1. Ensure Python 3.10+ is configured
2. Setup the virtualenv using ./setup-env
3. Run molecule test or the following commands (in order):

molecule create
molecule converge
molecule verify

Expected behavior

Molecule executes successfully without errors

Any helpful log output or screenshots

..snip..

INFO     Running default > verify
INFO     Executing Testinfra tests found in /tmp/ansible-role-amazon-ssm-agent/molecule/default/tests/...
ERROR: found no collectors for /tmp/ansible-role-amazon-ssm-agent/molecule/default/tests/test_default.py

============================= test session starts ==============================
platform linux -- Python 3.10.9, pytest-8.1.1, pluggy-1.5.0
rootdir: /
plugins: testinfra-10.1.0
collected 0 items / 1 error

==================================== ERRORS ====================================
______________________________ ERROR collecting . ______________________________
/opt/pyenv/versions/ansible-role-amazon-ssm-agent/lib/python3.10/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
/opt/pyenv/versions/ansible-role-amazon-ssm-agent/lib/python3.10/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/opt/pyenv/versions/ansible-role-amazon-ssm-agent/lib/python3.10/site-packages/_pytest/python.py:212: in pytest_collect_directory
    if pkginit.is_file():
/opt/pyenv/versions/3.10.9/lib/python3.10/pathlib.py:1322: in is_file
    return S_ISREG(self.stat().st_mode)
/opt/pyenv/versions/3.10.9/lib/python3.10/pathlib.py:1097: in stat
    return self._accessor.stat(self, follow_symlinks=follow_symlinks)
E   PermissionError: [Errno 13] Permission denied: '/lost+found/__init__.py'
=========================== short test summary info ============================
ERROR ../../../.. - PermissionError: [Errno 13] Permission denied: '/lost+fou...
=============================== 1 error in 0.08s ===============================

..snip..

Proposed Solution

According to this output, pytest isn't configured to set the rootdir, and the default of / is used instead. Touching pytest.ini in the root directory of this repo seems to resolve the issue.

[jsf9k]

I can't reproduce this with Python 3.12.3 or 3.10.9. I set up the Python virtual environment with, e.g.:

$ pyenv virtualenv 3.10.9 ansible-role-amazon-ssm-agent
$ pyenv local ansible-role-amazon-ssm-agent
$ pip install --upgrade pip setuptools wheel && pip install --upgrade --upgrade-strategy=eager -r requirements-dev.txt && pre-commit install

(This is analogous to what the setup-env script does.) Then I run molecule test and witness no issues.

@0xdeadbeefJERKY - Can you help me to reproduce the behavior you are seeing? From your output it does not appear that you are working from a Python local environment, as described in CONTRIBUTING.md.

[0xdeadbeefJERKY]

@jsf9k I just reproduced the issue again by doing the following:

git clone https://github.com/cisagov/ansible-role-amazon-ssm-agent
cd  ansible-role-amazon-ssm-agent
pyenv global 3.10
./setup-env
molecule test

This is being done from an Ubuntu 22.04 machine. I noticed that pytest-testinfra isn't being pinned to a specific version. Maybe there were some behavioral changes introduced in a more recent version?

❯ ansible --version     
ansible [core 2.16.2]
  config file = None
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/pyenv/versions/ansible-role-amazon-ssm-agent/lib/python3.10/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/pyenv/versions/ansible-role-amazon-ssm-agent/bin/ansible
  python version = 3.10.9 (main, Apr 23 2024, 15:25:11) [GCC 11.4.0] (/opt/pyenv/versions/ansible-role-amazon-ssm-agent/bin/python3)
  jinja version = 3.1.3
  libyaml = True

❯ molecule --version 
molecule 24.2.1 using python 3.10 
    ansible:2.16.2
    azure:23.5.3 from molecule_plugins
    containers:23.5.3 from molecule_plugins requiring collections: ansible.posix>=1.3.0 community.docker>=1.9.1 containers.podman>=1.8.1
    default:24.2.1 from molecule
    docker:23.5.3 from molecule_plugins requiring collections: community.docker>=3.4.11 ansible.posix>=1.4.0
    ec2:23.5.3 from molecule_plugins
    gce:23.5.3 from molecule_plugins requiring collections: google.cloud>=1.0.2 community.crypto>=1.8.0
    openstack:23.5.3 from molecule_plugins requiring collections: openstack.cloud>=2.1.0
    podman:23.5.3 from molecule_plugins requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
    vagrant:23.5.3 from molecule_plugins

❯ pytest --version                     
pytest 8.1.1