search

ansible / pytest-ansible

A pytest plugin that enables the use of ansible in tests, enables the use of pytest as a collection unit test runner, and exposes molecule scenarios through a pytest fixture.
https://ansible.readthedocs.io/projects/pytest-ansible/
MIT License
176 stars 64 forks source link

duplicates molecule tests after copying collection #292

Open evgeni opened 6 months ago

evgeni commented 6 months ago

In my collection I define the molecule tests like this (pretty much copy paste from the readme):

% cat tests/test_molecule.py
"""Tests for molecule scenarios."""
from __future__ import absolute_import, division, print_function

from pytest_ansible.molecule import MoleculeScenario

def test_molecule(molecule_scenario: MoleculeScenario) -> None:
    """Run molecule for each scenario.

    :param molecule_scenario: The molecule scenario object
    """
    proc = molecule_scenario.test()
    assert proc.returncode == 0

Now, when the testsuite runs, it seems to copy the collection to collections/ansible_collections/<namespace>/<name>, resulting in the following output:

============================= test session starts ==============================
platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0 -- /opt/hostedtoolcache/Python/3.11.8/x64/bin/python
cachedir: .pytest_cache
ansible: 2.15.10.post0
rootdir: /home/runner/work/namespace.collection/namespace.collection
plugins: ansible-24.1.2
collecting ... collected 10 items
collections/ansible_collections/namespace/collection/tests/test_molecule.py::test_molecule[repository-redhat] PASSED [ 10%]
collections/ansible_collections/namespace/collection/tests/test_molecule.py::test_molecule[repository-debian] PASSED [ 20%]
collections/ansible_collections/namespace/collection/tests/test_molecule.py::test_molecule[repository-suse] PASSED [ 30%]
collections/ansible_collections/namespace/collection/tests/test_molecule.py::test_molecule[agent-redhat] PASSED [ 40%]
collections/ansible_collections/namespace/collection/tests/test_molecule.py::test_molecule[agent-debian] PASSED [ 50%]
tests/test_molecule.py::test_molecule[repository-redhat] <- collections/ansible_collections/namespace/collection/tests/test_molecule.py PASSED [ 60%]
tests/test_molecule.py::test_molecule[repository-debian] <- collections/ansible_collections/namespace/collection/tests/test_molecule.py PASSED [ 70%]
tests/test_molecule.py::test_molecule[repository-suse] <- collections/ansible_collections/namespace/collection/tests/test_molecule.py PASSED [ 80%]
tests/test_molecule.py::test_molecule[agent-redhat] <- collections/ansible_collections/namespace/collection/tests/test_molecule.py PASSED [ 90%]
tests/test_molecule.py::test_molecule[agent-debian] <- collections/ansible_collections/namespace/collection/tests/test_molecule.py PASSED [100%]
======================== 10 passed in 717.30s (0:11:57) ========================

See how the tests are duplicated for collections/ansible_collections/namespace/collection/tests/test_molecule.py and tests/test_molecule.py?

Additionally, re-running the tests now fails (when executed on a non-ephemeral node, like a laptop):

=============================================================================================================== ERRORS ================================================================================================================
_______________________________________________________________________________________________ ERROR collecting tests/test_molecule.py _______________________________________________________________________________________________
import file mismatch:
imported module 'test_molecule' has this __file__ attribute:
  /home/evgeni/Devel/namespace.collection/collections/ansible_collections/namespace/collection/tests/test_molecule.py
which is not the same as the test file we want to collect:
  /home/evgeni/Devel/namespace.collection/tests/test_molecule.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
======================================================================================================= short test summary info =======================================================================================================
ERROR tests/test_molecule.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================================================================== 1 error in 0.08s ===========================================================================================================
evgeni commented 6 months ago

The same happens when using pytest --molecule:

============================= test session starts ==============================
platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0 -- /opt/hostedtoolcache/Python/3.11.8/x64/bin/python
cachedir: .pytest_cache
ansible: 2.16.5.post0
rootdir: /home/runner/work/namespace.collection/namespace.collection
plugins: ansible-24.1.2
collecting ... collected 10 items
collections/ansible_collections/namespace/collection/roles/agent/molecule/debian/molecule.yml::test0 PASSED [ 10%]
collections/ansible_collections/namespace/collection/roles/agent/molecule/redhat/molecule.yml::test1 PASSED [ 20%]
collections/ansible_collections/namespace/collection/roles/repository/molecule/debian/molecule.yml::test2 PASSED [ 30%]
collections/ansible_collections/namespace/collection/roles/repository/molecule/redhat/molecule.yml::test3 PASSED [ 40%]
collections/ansible_collections/namespace/collection/roles/repository/molecule/suse/molecule.yml::test4 PASSED [ 50%]
roles/agent/molecule/debian/molecule.yml::test5 PASSED                   [ 60%]
roles/agent/molecule/redhat/molecule.yml::test6 PASSED                   [ 70%]
roles/repository/molecule/debian/molecule.yml::test7 PASSED              [ 80%]
roles/repository/molecule/redhat/molecule.yml::test8 PASSED              [ 90%]
roles/repository/molecule/suse/molecule.yml::test9 PASSED                [100%]
=============================== warnings summary ===============================