Ansible lint thinks molecule task files are playbooks

[antivirtel]

SUMMARY

I've opened a bug at ansible-lint for the same problem. Can you please agree with them which structure molecule should be? I think molecule should be as is, all molecule related files under <repo_root>/roles/<rolename>/molecule/, but I don't think they agree: https://github.com/ansible-community/ansible-lint/issues/1622#issuecomment-860081793

This says nothing about molecule or tests: https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#role-directory-structure so I think the whole thing should be kept together under <repo_root>/roles/<rolename>/molecule, shouldn't it?

Problem:

We're using the latest Molecule (v3.3.4) to test our Ansible roles. Some roles require a full VM, so we're using Molecule tasks to create them. Ansible-lint latest thinks they're playbook files, but they should be considered as test files. Molecule puts their files to <repo_root>/roles/<rolename>/molecule/<scenario:default>. I know it should be under according to the collection layout: <repo_root>/tests/molecule or <repo_root>/roles/<rolename>/tests/molecule/<scenario:default>, but this is how it is generated still by molecule...

The molecule itself will generate the files like create_linux_instance.yml. To generate an example file, install molecule, go to your roles dir (<repo_root>/roles/<rolename>), then execute: molecule init scenario default -d vmware.

ISSUE TYPE

• Bug Report

ANSIBLE VERSION

$ ansible --version
ansible 2.9.22
  config file = ~/deploy/<repo_root>/ansible.cfg
  configured module search path = ['~/deploy/<repo_root>/library']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.8 (default, Nov 16 2020, 16:55:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

$ ansible-lint --version
ansible-lint 5.0.12 using ansible 2.9.22

$ molecule --version
3.3.4

CONFIGURATION

molecule init scenario default -d vmware

OS / ENVIRONMENT

$ uname -a
Linux 3.10.0-1127.18.2.el7.x86_64 #1 SMP Sun Jul 26 15:27:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)

STEPS TO REPRODUCE

To generate an example file, install molecule, go to any roles dir (/roles/), then execute: molecule init scenario default -d vmware.

Snippet (register is used multiple times after):

- name: Create keypair
  user:
    name: "{{ lookup('env','USER') | default('root', true) }}"
    generate_ssh_key: true
    ssh_key_file: "{{ keypair_path }}"
  register: keypair_result

EXPECTED RESULTS

Molecule files interpreted as test/task files, not playbooks.

ACTUAL RESULTS

$ ansible-lint -vvvv -c .ansible-lint roles
DEBUG    Logging initialized to level 10
DEBUG    Options: Namespace(cache_dir='~/.cache/ansible-lint/ccc58c', colored=True, config_file='~/deploy/<rep_root>/.ansible-lint', configured=True, cwd=PosixPath('~/deploy/<rep_root>'), display_relative_path=True, enable_list=[], exclude_paths=['.cache', '.git', '.hg', '.svn', '.tox'], extra_vars={'user_defined_hosts': ['localhost'], 'adhoc_ip_address': '127.0.0.1', 'inventory_target': 'localhost'}, format='rich', kinds=[{'jinja2': '**/*.j2'}, {'jinja2': '**/*.j2.*'}, {'requirements': '**/meta/requirements.yml'}, {'galaxy': '**/galaxy.yml'}, {'reno': '**/releasenotes/*/*.{yaml,yml}'}, {'playbook': '**/playbooks/*.{yml,yaml}'}, {'playbook': '**/*playbook*.{yml,yaml}'}, {'role': '**/roles/*/'}, {'tasks': '**/tasks/**/*.{yaml,yml}'}, {'handlers': '**/handlers/*.{yaml,yml}'}, {'vars': '**/{host_vars,group_vars,vars,defaults}/**/*.{yaml,yml}'}, {'meta': '**/meta/main.{yaml,yml}'}, {'yaml': '.config/molecule/config.{yaml,yml}'}, {'requirements': '**/molecule/*/{collections,requirements}.{yaml,yml}'}, {'yaml': '**/molecule/*/{base,molecule}.{yaml,yml}'}, {'requirements': '**/requirements.yml'}, {'playbook': '**/molecule/*/*.{yaml,yml}'}, {'yaml': '**/{.ansible-lint,.yamllint}'}, {'yaml': '**/*.{yaml,yml}'}, {'yaml': '**/.*.{yaml,yml}'}], lintables=['roles'], listrules=False, listtags=False, loop_var_prefix=None, mock_modules=[], mock_roles=[], offline=False, parseable=False, parseable_severity=False, progressive=False, project_dir='.', quiet=False, rules={}, rulesdir=[], rulesdirs=['/usr/local/lib/python3.6/site-packages/ansiblelint/rules'], skip_action_validation=True, skip_list=['yaml', '204', 'var-spacing', 'risky-file-permissions', 'risky-file-permissions', 'package-latest', 'package-latest', 'partial-become', 'partial-become', 'empty-string-compare', 'empty-string-compare', 'meta-no-info', 'meta-no-info', 'experimental'], tags=[], use_default_rules=False, var_naming_pattern='^[a-z_][a-z0-9_]*$', verbosity=4, version=False, warn_list=['experimental', 'role-name'])
INFO     Added ANSIBLE_ROLES_PATH=~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:roles:roles
DEBUG    Loading rules from /usr/local/lib/python3.6/site-packages/ansiblelint/rules
INFO     Added ANSIBLE_ROLES_PATH=~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:roles:roles
Loading custom .yamllint config file, this extends our internal yamllint config.
INFO     Discovered files to lint using: git ls-files -z
INFO     Executing syntax check on roles/<rolename>/molecule/default/create_linux_instance.yml (3.03s)
[... a long list of files here...]
INFO     Executing syntax check on roles/grafana/molecule/default/converge.yml (3.44s)
WARNING  Listing 4 violation(s) that are fatal
syntax-check: 'register' is not a valid attribute for a Play
roles/<rolename>/molecule/default/create_linux_instance.yml:2:3 ERROR! 'register' is not a valid attribute for a Play

The error appears to be in '~/deploy/<rep_root>/roles/docker/molecule/default/create_linux_instance.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- name: Create keypair
  ^ here

[... 3 more ...]

[sky-joker]

Thanks, @antivirtel for reporting the issue. I tried whether the same error occurs, but the error didn't occur with the following procedure.

$ python3 -m venv venv
$ . venv/bin/activate
(venv)$ pip install --upgrade pip
(venv)$ pip install molecule==3.3.4 molecule-vmware ansible==2.9.22
(venv)$ pip list | grep -e 'Pack' -e '---' -e 'molecule' -e 'ansible'
Package            Version
------------------ ---------
ansible            2.9.22
ansible-lint       5.0.12
molecule           3.3.4
molecule-vmware    0.3.5
(venv)$ mkdir -p roles/sample/tasks
(venv)$ touch roles/sample/tasks/main.yml
(venv)$ cd roles/sample/
(venv)$ molecule init scenario -d vmware
(venv)$ find . -type f
./tasks/main.yml
./molecule/default/molecule.yml
./molecule/default/INSTALL.rst
./molecule/default/converge.yml
./molecule/default/create.yml
./molecule/default/create_linux_instance.yml
./molecule/default/create_windows_instance.yml
./molecule/default/destroy.yml
./molecule/default/verify.yml
./.yamllint
(venv)$ cd ../../
(venv)$ ls
roles  venv
(venv)$ ansible-lint roles

Do you know what I'm doing wrong?

[antivirtel]

Do you have register: in ./molecule/default/create_linux_instance.yml?

[sky-joker]

Do you have register: in ./molecule/default/create_linux_instance.yml?

Yes. I'm wondering why the same error didn't occur in my environment🤔

(venv)$ cat roles/sample/molecule/default/create_linux_instance.yml

create_linux_instance.yml

---
- name: Create keypair
  user:
    name: "{{ lookup('env','USER') }}"
    generate_ssh_key: true
    ssh_key_file: "{{ keypair_path }}"
  register: keypair_result

- name: Create molecule instance(s)
  vmware_guest:
    hostname: "{{ molecule_yml.driver.vcenter_hostname }}"
    username: "{{ molecule_yml.driver.vcenter_username }}"
    password: "{{ molecule_yml.driver.vcenter_password }}"
    validate_certs: "{{ molecule_yml.driver.validate_certs | default(false) }}"
    datacenter: "{{ molecule_yml.driver.datacenter }}"
    cluster: "{{ molecule_yml.driver.cluster | default(omit) }}"
    esxi_hostname: "{{ molecule_yml.driver.esxi_hostname | default(omit) }}"
    folder: "{{ molecule_yml.driver.folder }}"
    name: "{{ item.name }}"
    template: "{{ item.template }}"
    snapshot_src: "{{ item.snapshot_src | default(omit) }}"
    hardware: "{{ item.hardware | default(omit) }}"
    networks: "{{ item.networks }}"
    customization:
      hostname: "{{ item.name }}"
      password: "{{ molecule_yml.driver.vm_password }}"
      dns_servers: "{{ molecule_yml.driver.dns_servers | default(omit) }}"
    wait_for_ip_address: true
    wait_for_customization: true
    state: poweredon
  async: 7200
  poll: 0
  register: async_results
  loop: "{{ molecule_yml.platforms }}"

- name: Check the test instance creation status
  async_status:
    jid: "{{ item.ansible_job_id }}"
  loop: "{{ async_results.results }}"
  register: server
  until: server.finished
  delay: 10
  retries: 300

- name: Check vmware tools status
  vmware_guest_info:
    hostname: "{{ molecule_yml.driver.vcenter_hostname }}"
    username: "{{ molecule_yml.driver.vcenter_username }}"
    password: "{{ molecule_yml.driver.vcenter_password }}"
    validate_certs: "{{ molecule_yml.driver.validate_certs | default(false) }}"
    datacenter: "{{ molecule_yml.driver.datacenter }}"
    folder: "{{ molecule_yml.driver.folder }}"
    name: "{{ item.name }}"
  register: guest_state
  until: "guest_state.instance.guest_tools_status == 'guestToolsRunning'"
  delay: 10
  retries: 300
  loop: "{{ molecule_yml.platforms }}"
  ignore_errors: true

- name: Create ssh directory and public key in guest OS
  vmware_vm_shell:
    hostname: "{{ molecule_yml.driver.vcenter_hostname }}"
    username: "{{ molecule_yml.driver.vcenter_username }}"
    password: "{{ molecule_yml.driver.vcenter_password }}"
    validate_certs: "{{ molecule_yml.driver.validate_certs | default(false) }}"
    datacenter: "{{ molecule_yml.driver.datacenter }}"
    folder: "{{ molecule_yml.driver.folder }}"
    vm_id: "{{ item.name }}"
    vm_username: "{{ molecule_yml.driver.vm_username }}"
    vm_password: "{{ molecule_yml.driver.vm_password }}"
    vm_shell: /bin/bash
    vm_shell_args: "-c 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo {{ keypair_result.ssh_public_key }} > ~/.ssh/authorized_keys'"
    wait_for_process: true
  loop: "{{ molecule_yml.platforms }}"

- name: Populate instance config dict
  set_fact:
    instance_conf_dict: {
      'instance': "{{ instance_info.item.item.name }}",
      'address': "{{ instance_info.instance.ipv4 }}",
      'user': "{{ molecule_yml.driver.vm_username }}",
      'port': 22,
      'identity_file': "{{ keypair_path }}",
      'instance_os_type': "{{ molecule_yml.driver.instance_os_type }}"
    }
  loop: "{{ server.results }}"
  loop_control:
    loop_var: instance_info
  register: instance_config_dict
  when: server is changed

- name: Convert instance config dict to a list.
  set_fact:
    instance_conf: "{{ instance_config_dict.results | map(attribute='ansible_facts.instance_conf_dict') | list }}"
  when: server is changed

- name: Dump instance config
  copy:
    content: "{{ instance_conf | to_json | from_json | to_yaml }}"
    dest: "{{ molecule_instance_config }}"
    mode: 0644
  when: server is changed

[bandit420]

Can confirm same issue.

pip list | grep -e 'Pack' -e '---' -e 'molecule' -e 'ansible'
Package           Version
----------------- ---------
ansible           3.4.0
ansible-base      2.10.10
ansible-lint      5.0.12
molecule          3.3.4
molecule-docker   0.2.4
molecule-goss     1.1
molecule-vmware   0.3.5

As a (ugly) workaround you can ignore paths:

Create .ansible-lint config file and ignore the paths (https://ansible-lint.readthedocs.io/en/latest/configuring.html#configuration-file).

exclude_paths:
- molecule/*/create_linux_instance.yml
- molecule/*/create_windows_instance.yml

At least "molecule lint" is then working for the moment.

[antivirtel]

Ansible-lint runs in a Docker container for us, so no venv/pyenv needed there. For the config file .ansible-lint the config exclude_paths: only works without the *, so each role with Vmware testing needs to be listed. So like this:

exclude_paths:
- roles/<rolename1>/molecule
- roles/<rolename2>/molecule

They suggested a fix: https://github.com/ansible-community/ansible-lint/issues/1622#issuecomment-860090106 - would that be OK?

[sky-joker]

@antivirtel

I want to confirm that the error willn't happen by changing the directory structure to fix the issue, but unfortunately, I can't confirm the error in my environment.
I want to make sure that can you check the error not happen in your environment by adopting the suggestion?

[antivirtel]

$ pip3 list | grep -e 'Pack' -e '---' -e 'molecule' -e 'ansible'
Package             Version
------------------- ---------
ansible             2.9.22
ansible-lint        5.0.12
molecule            3.3.4
molecule-docker     0.2.4
molecule-vmware     0.3.5

Before:

$ tree roles/<rolename>/molecule/
roles/<rolename>/molecule/
└── default
    ├── converge.yml
    ├── create_linux_instance.yml
    ├── create.yml
    ├── destroy.yml
    └── molecule.yml

$ grep create_linux roles/<rolename>/molecule/default/create.yml
    - include_tasks: create_linux_instance.yml

Output before

$ ansible-lint roles
Loading custom .yamllint config file, this extends our internal yamllint config.
WARNING  Listing 4 violation(s) that are fatal
[...3 more...]

syntax-check: 'register' is not a valid attribute for a Play
roles/<rolename>/molecule/default/create_linux_instance.yml:2:3 ERROR! 'register' is not a valid attribute for a Play

The error appears to be in '/home/rbk03/deploy/provisioning/roles/<rolename>/molecule/default/create_linux_instance.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- name: Create keypair
  ^ here

Changing:

$ cd roles/<rolename>/molecule/default/
$ mkdir tasks
$ mv create_* tasks/
$ cd -
$ tree roles/<rolename>/molecule/
roles/<rolename>/molecule/
└── default
    ├── converge.yml
    ├── create.yml
    ├── destroy.yml
    ├── molecule.yml
    └── tasks
        └── create_linux_instance.yml

2 directories, 5 files

Tried to modify the path to the task, but doesn't seem to accept it (nor on the original):

$ grep create_linux roles/<rolename>/molecule/default/create.yml
    - include_tasks: ./tasks/create_linux_instance.yml

or these:

$ grep create_linux roles/<rolename>/molecule/default/create.yml
    - include_tasks: tasks/create_linux_instance.yml

$ grep create_linux roles/<rolename>/molecule/default/create.yml
    - include_tasks: ../default/tasks/create_linux_instance.yml

$ grep create_linux roles/<rolename>/molecule/default/create.yml
    - include_tasks: ../<rolename>/molecule/default/tasks/create_linux_instance.yml

$ grep -B1 create_linux roles/<rolename>/molecule/default/create.yml
    - include_tasks:
        file: tasks/create_linux_instance.yml

I always get this error, like the above setting is ignored:

$ ansible-lint roles
Loading custom .yamllint config file, this extends our internal yamllint config.
WARNING  Listing 1 violation(s) that are fatal
internal-error: Unexpected error code 1 from execution of: ansible-playbook --syntax-check --extra-vars {"user_defined_hosts": ["localhost"], "adhoc_ip_address": "127.0.0.1", "inventory_target": "localhost"} roles/<rolename>/molecule/default/create_linux_instance.yml
roles/<rolename>/molecule/default/create_linux_instance.yml:1 ERROR! the playbook: roles/<rolename>/molecule/default/create_linux_instance.yml could not be found

You can skip specific rules or tags by adding them to your configuration file:
# .ansible-lint
warn_list:  # or 'skip_list' to silence them completely
  - internal-error  # Unexpected internal error
Finished with 1 failure(s), 0 warning(s) on 1378 files.

What's the correct line there? I've asked on the other ticket as well: https://github.com/ansible-community/ansible-lint/issues/1622#issuecomment-861655733

[sky-joker]

@antivirtel

Thanks for making time to check that.
By using docker, I could confirm the same error.

ok.
I agree that moves the task files to the tasks directory to fix the issue.
I'm probably going to take some time to make a patch for fixing the error.

[sky-joker]

@antivirtel

I made a patch(https://github.com/sky-joker/molecule-vmware/pull/21) to fix the issue, so could you please check the error will not happen in your environment?
You can do check with the following procedure.

$ git clone https://github.com/sky-joker/molecule-vmware.git
$ cd molecule-vmware/
$ git checkout -b fix_ansiblelint5_error remotes/origin/fix_ansiblelint5_error
$ pip install .
$ pip install ansible==2.9.22
$ cd ../
$ mkdir -p roles/sample
$ cd roles/sample
$ molecule init scenario -d vmware
$ cd ../../
$ ls
molecule-vmware  roles
$ ansible-lint --exclude molecule-vmware . 

And let me know the result.

[antivirtel]

Thank you for the quick fix!

After the patched version:

$ tree molecule/default/
molecule/default/
├── converge.yml
├── create.yml
├── destroy.yml
├── handlers
│   └── main.yml
├── INSTALL.rst
├── molecule.yml
├── tasks
│   ├── create_linux_instance.yml
│   └── create_windows_instance.yml
└── verify.yml

2 directories, 9 files

Looks ok now, it revealed our other errors (so gone through this step, issue with Playbooks in Molecule). How can we migrate the current roles (which were already inited)? What exactly do I need to put into the include_tasks: mentioned here?

[sky-joker]

Thank you for checking driver behavior.

About Migration

You can be migrated to copy molecule.yml from the old molecule scenario directory to a new one after generating the new scenario.
I made a sample script to migrate.
Please refer to:

https://gist.github.com/sky-joker/a5e989367cb3e559dc3bc94357018110

How to use the above tool

Please execute in the same hierarchy of the roles directory.

$ ls
migration_molecule_vmware.py  roles  venv
$ ./migration_molecule_vmware.py

If you will not specify the delete option, molecule backup directry will be created.

$ ls roles/sample/
molecule  molecule_back  tasks

$ find roles/sample/molecule -type f
roles/sample/molecule/default/molecule.yml
roles/sample/molecule/default/tasks/main.yml
roles/sample/molecule/default/tasks/create_linux_instance.yml
roles/sample/molecule/default/tasks/create_windows_instance.yml
roles/sample/molecule/default/handlers/main.yml
roles/sample/molecule/default/INSTALL.rst
roles/sample/molecule/default/converge.yml
roles/sample/molecule/default/create.yml
roles/sample/molecule/default/destroy.yml
roles/sample/molecule/default/verify.yml

$ find roles/sample/molecule_back/ -type f
roles/sample/molecule_back/default/molecule.yml
roles/sample/molecule_back/default/tasks/main.yml
roles/sample/molecule_back/default/INSTALL.rst
roles/sample/molecule_back/default/converge.yml
roles/sample/molecule_back/default/create.yml
roles/sample/molecule_back/default/create_linux_instance.yml
roles/sample/molecule_back/default/create_windows_instance.yml
roles/sample/molecule_back/default/destroy.yml
roles/sample/molecule_back/default/verify.yml

$ openssl md5 roles/sample/molecule_back/default/molecule.yml
MD5(roles/sample/molecule_back/default/molecule.yml)= 0feb8fd208ffe4c80d97040b97cf5108
$ openssl md5 roles/sample/molecule/default/molecule.yml
MD5(roles/sample/molecule/default/molecule.yml)= 0feb8fd208ffe4c80d97040b97cf5108