Allow for tasks and vars files distribution through collections

[NoSuchCommand]

Proposal: Allow for tasks and vars files distribution through collections

Author: Fabien Malfoy <@NoSuchCommand>

Date: 2021-12-24 (HO! HO! HO!)

• Status: New
• Proposal type: core design
• Targeted release: 2.13
• Estimated time to implement: N/A

Motivation

The current (ansible-core 2.12) implementation of Ansible collections allows them to distribute many types of content like roles, plugins of various types, or even playbooks. In the latter case, the playbooks can easily be referenced by the end-user using their FQCN on the command line or with an import_playbook directive. Unfortunately, a playbook embeds some hardcoded informations like the hosts, gather_facts, and many others, on which the end-user might want to keep control.

In many cases, a collection developer might only want to distribute intelligence by means of just bare tasks files rather than playbooks, thus letting the control of the running context details to the future consumer of the collection.

The same suggestion also applies to vars files, for variables that concern the collection globally.

Problems

What problems exist that this proposal will solve?

No auto-discovery mechanism of tasks nor variables within collections

While the import_playbook directive allows for auto-discovery of content in the playbooks/ subdirectory of a collection with an FQCN, there is no such mechanism for tasks with import_tasks/include_tasks, nor variables with vars_files/include_vars.

Solution proposal

Collections could optionally contain a /tasks and /vars directory to host tasks files and vars files respectively. These could easily be referenced with their FQCN when found in front of context-dependent directives like import_tasks/include_tasks or vars_files/include_vars.

---
# Third-party playbook written by the end-user in her project

- hosts: my_pattern # As the end-user, I keep control of the target pattern
  gather_facts: no  # and other modalities like facts gathering
  serial: 3         # or even the strategy and its options

  pre_tasks:
  # This would match /vars/foobar.yml inside mynamespace.mycollection
  - include_vars: mynamespace.mycollection.foobar

  # Here, I am free to insert any additional 3rd party intelligence
  # I want, and even a static `roles:` section.

  tasks:

  # Here, again, I can execute some tasks of my own before
  # calling the tasks from my collection.

  # This would import /tasks/some_tasks.yml from the collection
  - import_tasks: mynamespace.mycollection.some_tasks

  # Moreover, looping over blocks of tasks can only be done with tasks files
  - include_tasks: mynamespace.mycollection.recurring_tasks
    loop: "{{ mylist }}"

In addition, these directories would serve as secondary lookup location when importing or including tasks/variables from within roles of the same collection. The collections: directive would simply add a least-precedence level search path.

---
# Some playbook

- hosts: my_pattern
  collections:
  - mynamespace.mycollection

  tasks:
  # Search is relative to the playbook dir first, then /vars/foobar.yml from the collection
  - include_vars: foobar.yml

  # Same for tasks, playbook dir first, then /tasks/some_tasks.yml from the collection
  - iimport_tasks: some_tasks.yml

Within 3rd party roles:

# some 3rd-party role's meta/main.yml
# ...
collections:
- mynamespace.mycollection
# ...

# some 3rd-party role's tasks/main.yml

# Search is role's tasks/ directory first, then /tasks/some_tasks.yml
# from the collection (because of the meta/main.yml above)
- import_tasks: some_tasks.yml

Within roles of the same collection, the collections: statement in meta/main.yml has an implicit reference to the containing collection as the first element of the list.

# mynamespace.mycollection.my_role meta/main.yml
# ...
collections:
- somenamespace.othercollection
# ...

# mynamespace.mycollection.my_role tasks/main.yml

# Search is:
#   - role's tasks/ directory first
#   - then /tasks/some_tasks.yml from the containing collection (implicit)
#   - then /tasks/some_tasks.yml from somenamespace.othercollection (because of the meta/main.yml above)
- import_tasks: some_tasks.yml

Documentation

This proposal propably involves modifying the behaviour of core modules like import_tasks, include_tasks, include_vars. Their documentation should be adjusted accordingly.

[agaffney]

How is this really different than what you can already accomplish by including/importing a role distributed via a collection? Roles can contain both tasks files and vars files, and you can easily control which particular tasks file that you use from a role.

[bcoca]

No auto-discovery mechanism of tasks nor variables within collections

Not really true, there is no DIRECT discovery, but using a 'collection playbook' or role has 'relative discovery' to the role or playbook, so they can use the include/import_tasks/vars against the collection.

[briantist]

I would be more interested in bare vars than tasks. I would use these mostly with --extra-vars @namespace.collection.vars_file, as a way of grouping certain sets of possible options.

I use this pattern now a directory within local projects, but I would find value in being able to package them in collections.

I think if the loose task files thing existed though, I'd discourage its use in favor of roles, personally.