molecule test does not pickup role

Prerequisites

[X] This was not already reported in the past (duplicate check)
[X] It does reproduce it with code from main branch (latest unreleased version)
[X] I include a minimal example for reproducing the bug
[X] The bug is not trivial, as for those a direct pull-request is preferred
[X] Running pip check does not report any conflicts
[X] I was able to reproduce the issue on a different machine
[X] The issue is not specific to any driver other than 'default' one

Environment

$ molecule --version
molecule 24.7.0 using python 3.12
    ansible:2.17.1
    azure:23.5.3 from molecule_plugins
    containers:23.5.3 from molecule_plugins requiring collections: ansible.posix>=1.3.0 community.docker>=1.9.1 containers.podman>=1.8.1
    default:24.7.0 from molecule
    docker:23.5.3 from molecule_plugins requiring collections: community.docker>=3.4.11 ansible.posix>=1.4.0
    ec2:23.5.3 from molecule_plugins
    gce:23.5.3 from molecule_plugins requiring collections: google.cloud>=1.0.2 community.crypto>=1.8.0
    openstack:23.5.3 from molecule_plugins requiring collections: openstack.cloud>=2.1.0
    podman:23.5.3 from molecule_plugins requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
    vagrant:23.5.3 from molecule_plugins

What happened

I have role which support (or should support) different distributions.

The role is created with the command ansible-galaxy init base

let's assume the role only has the following task in tasks/main.yml

- name: Install and setup Postfix
  tags: postfix
  block:
    - name: Install Dependencies
      become: true
      ansible.builtin.package:
        name:
          - postfix
          - libsasl2-modules
          - bsd-mailx
          - mailutils
        state: present

There are a few more tasks, but this task will fail on different distributions.

Further, I've added the molecule scenario with molecule init scenario while I was in the role directory.

The final structure on the role looks like the following:

├── base
│   ├── defaults
│   │   └── main.yml
│   ├── handlers
│   │   └── main.yml
│   ├── meta
│   │   └── main.yml
│   ├── README.md
│   ├── tasks
│   │   └── main.yml
│   ├── templates
│   ├── tests
│   │   ├── inventory
│   │   └── test.yml
│   └── vars
│       └── main.yml
├── molecule
│   └── default
│       ├── converge.yml
│       ├── create.yml
│       ├── destroy.yml
│       ├── molecule.yml
│       └── tasks
│           └── create-fail.yml
├── README.md
└── VERSION

When running the command molecule test It seems the role is getting included, but it never runs and so it does also not fail on the rockylinux8 docker.

PLAY [Converge] ****************************************************************

TASK [Check uname] *************************************************************
ok: [ubuntu2404]
ok: [rocky8]

TASK [Print some info] *********************************************************
ok: [rocky8] => {
    "changed": false,
    "msg": "All assertions passed"
}
ok: [ubuntu2404] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Include base role] *******************************************************
included: base for rocky8, ubuntu2404

PLAY RECAP *********************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
rocky8                     : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ubuntu2404                 : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Using instead:

roles:
    - base

it will never be called.

Reproducing example

molecule.yml:

---
driver:
  name: docker
platforms:
  - name: rocky8
    image: rockylinux:8-minimal
  - name: ubuntu2404
    image: ubuntu:24.04
provisioner:
  name: ansible
  scenario:
    name: default
verifier:
  name: ansible
    # you might want to add your own variables here based on what provisioning
    # you are doing like:
    # image: quay.io/centos/centos:stream8

converge.yml:

- name: Fail if molecule group is missing
  hosts: localhost
  tasks:
    - name: Print some info
      ansible.builtin.debug:
        msg: "{{ groups }}"

    - name: Assert group existence
      ansible.builtin.assert:
        that: "'molecule' in groups"
        fail_msg: |
          molecule group was not found inside inventory groups: {{ groups }}

- name: Converge
  hosts: molecule
  gather_facts: false
  tasks:
    - name: Check uname
      ansible.builtin.raw: uname -a
      register: result
      changed_when: false

    - name: Include base role
      ansible.builtin.include_role:
        name: base

Are you sure that Ansible itself is able to find the role? Try something similar to:

ssbarnea@m1: ~/c/a/ansible-lint/examples fix/working_directory
$ ansible -m import_role -a name=role_detection localhost
[WARNING]: No inventory was parsed, only implicit localhost is available

ssbarnea@m1: ~/c/a/ansible-lint/examples fix/working_directory
$ ansible -m import_role -a name=role_detection222 localhost
[WARNING]: No inventory was parsed, only implicit localhost is available
ERROR! the role 'role_detection222' was not found in ./roles:/Users/ssbarnea/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:.
FAIL: 1

If Ansible itself is not finding the role, is not a valid bug.

The result of ansible -m import_role -a name=base localhost from the root of my role repo is the following:

localhost | FAILED! => {
    "changed": false,
    "msg": "Failed to lookup user ansible: \"getpwnam(): name not found: 'ansible'\""
}

This seems fine, since the first task of the role adds ssh pub keys to the user "ansible", which is not available on my local machine.

So I run the command again and override the base_ssh_user variable:

ansible -m import_role -a name=base localhost -e base_ssh_user=$USER, which worked and now my local admins have access to my local machine ;)

@ssbarnea I am pretty sure molecule is finding the role since as soon as change the name to something else converge fails. I have just restarted on a fresh branch of my role implementing the scenario "docker" the same way the documentation is mentioning: molecule-docker

I just replaced the following:

  gather_facts: false
  tasks:
-    - name: Check uname
-      ansible.builtin.raw: uname -a
-      register: result
-      changed_when: false
-
-    - name: Print some info
-      ansible.builtin.assert:
-        that: result.stdout | regex_search("^Linux")
+    - name: Include base role
+      ansible.builtin.include_role:
+        name: base
+        tasks_from: main.yml

Same result as before. If I run any other task directly, the containers are executed as expected. Like replaced example.

If I use as rolename anything else, as an example: "baser", the task fails becaues the system does not find the role.

I have the exact same problem. I only have a ansible.builtin.include_role task in my converge.yml file. If I change the name of the role to one that doesn't exist, it will complain that role does not exists. When I run converge it will pass without any errors, but none of my tasks in the role will be executed.

$ molecule --version
molecule 24.9.0 using python 3.12
    ansible:2.17.4
    azure:23.5.3 from molecule_plugins
    containers:23.5.3 from molecule_plugins requiring collections: ansible.posix>=1.3.0 community.docker>=1.9.1 containers.podman>=1.8.1
    default:24.9.0 from molecule
    docker:23.5.3 from molecule_plugins requiring collections: community.docker>=3.4.11 ansible.posix>=1.4.0
    ec2:23.5.3 from molecule_plugins
    gce:23.5.3 from molecule_plugins requiring collections: google.cloud>=1.0.2 community.crypto>=1.8.0
    openstack:23.5.3 from molecule_plugins requiring collections: openstack.cloud>=2.1.0
    podman:23.5.3 from molecule_plugins requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
    vagrant:23.5.3 from molecule_plugins

ansible / molecule