Proposal: move cookiecutter code into proper ansible collection

[beargiles]

One of the biggest annoyances I have with the current approach is that it's basically impossible to upgrade the driver. It creates the scripts in the same location as our custom code (converge, verify, etc.) and a lot of us are weenies who don't to risk re-running a script nuking what we have.

(I know - we could probably commit, clean out the directory, re-run the script, merge, commit... but that's a lot of work.)

A better idea (IMHO) is to move the cookie cutter implementations into an Ansible collection. The molecule.yml file and stub prepare, create, and destroy scripts would remain but they would do nothing but import the corresponding tasks from the new collection.

This makes updates, e.g., for bug fixes, trivial. It would leave our custom scripts untouched.

This could be created as a separate project, or this repo could be modified with the goal of moving the existing python code into the collection's 'modules' directory with just a stub left for the 'molecule init --driver...' call. Among other things this would potentially allow that code to be reused since it would be available as, e.g., 'ansible_community.molecule_plugins.ec2_create_test_stub'.

[kksat]

we found a way around it. In cookie cutter templates we have (for instance for create action) following code

- name: Create
  ansible.builtin.import_playbook: <our namespace>.<our collection>.create 

At the same time in dependency section in molecule.yml we have following

---
dependency:
  name: shell
  # yamllint disable rule:line-length
  command: |
    ansible-galaxy install -r $MOLECULE_SCENARIO_DIRECTORY/requirements.yml
  # yamllint enable rule:line-length

This will install all collections, mentioned in requirements.yml (this file is also in molecule driver in cookie cutter) and collection

. is mentioned in requirements.yml With every run `molecule dependency` all dependent collections are installed. Hope this helps.

[apatard]

@beargiles I'm not sure to understand. Are you modifying the cookiecutter templates from inside the driver directory, leading to loss of changes after every update ? Can't you use provisioner.playbooks.create in molecule.yml ?

[beargiles]

I'll have a prototype soon - I've been adding a ton of verification, tracking things more carefully, etc.

The general idea is that the cookiecutter for create.yml and destroy.yml will become

---
{% raw -%}
---
- name: Create
  hosts: localhost
  connection: local
  gather_facts: false
  no_log: "{{ molecule_no_log }}"
  tasks:
    - ansible.builtin.include_role:
      name: snapteam.molecule_plugins.ec2_create
{%- endraw %}

and

---
{% raw -%}
---
- name: Destroy
  hosts: localhost
  connection: local
  gather_facts: false
  no_log: "{{ molecule_no_log }}"
  tasks:
    - ansible.builtin.include_role:
      name: snapteam.molecule_plugins.ec2_destroy
{%- endraw %}

('snapteam' is internal project - obviously it would be better to use ansible-community.something)

This uses a new collection 'snapteam.molecule-plugins' that's based on the current cookiecutter implementation but made into real roles. E.g., a lot of 'vars:' is replaced by ec2-create/vars/main.yml, etc.

It's entirely independent of the existing plugin - it simply moves two cookiecutter files into a new collection that can be independently maintained and updated. The existing module includes the code via 'include_role'.

(Note: I started with a single role but ultimately decided everything was much clearer after I split it into two roles and the old create.yml and destroy.yml became the prototypes for the new main.yml.)

.....

That said... one of the nice things about Collections is that they make it easy to include modules in addition to roles. There's no reason why the existing python code couldn't be converted into modules, e.g., 'ansible-community.molecule_plugins.ec2_install', with just a stub left so molecule can find the required configuration details and code.

In fact the more I look at it the more I think it may make sense to ultimately ditch the ansible implementration entirely. Since we only require a few functions, even with my extended validation, it should be much easier to maintain than my current account that requires quite a lot of 'register' and 'assert' plays.

If create and destroy are converted to python modules the cookiecutter code would be reduced to

---
{% raw -%}
---
- name: Create
  hosts: localhost
  connection: local
  gather_facts: false
  no_log: "{{ molecule_no_log }}"
  tasks:
    - snapteam.molecule_plugins.ec2_create
{%- endraw %}

Of course there would still be ways to support additional functionality. In this iteration my focus has been on moving as much as possible into a separate collection while maintaining sane defaults. You can do a lot if you're using a role and have a vars/main.yml file.