Running a role multiple times in one playbook doesn't work

[ghomem]

Summary

I am trying to reuse a role by executing it multiple times with different parameters like recommended in the documentation:

https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#running-a-role-multiple-times-in-one-playbook

Example from the documents:

- hosts: webservers
  roles:
    - { role: foo, vars: { message: "first" } }
    - { role: foo, vars: { message: "second" } }

Issue Type

Bug Report

Component Name

user

Ansible Version

ansible 2.9.6
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/remoteops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

Configuration

the output from theansible-config dump --only-changed  command is empty

OS / Environment

Ubuntu 20.04

Steps to Reproduce

roles:
    - node_base
    - node_extra_features
    - { role: base_ssl_domain,   vars: { my_prefix: 'star.staging.MYDOMAIN.TLD' } }
    - { role: base_nginx_secure, vars: { my_cleanup: false } }

    # this creates a static server
    - { role: base_nginx_static_domain,
        vars: { my_server_name: 'testing-srv.staging.MYDOMAIN.TLD',
              my_ssl_prefix : '{{ _my_ssl_prefix }}',
              my_root_folder: '/usr/share/nginx/html' } }

   # this creates another static server, root dir was created by hand
    - { role: base_nginx_static_domain,
        vars: { my_server_name: 'testing-srv.staging.MYDOMAIN.TLD',
                my_ssl_prefix : '{{ _my_ssl_prefix }}',
                my_root_folder: '/usr/share/nginx/html2' } }

    # this creates another static server, root dir was created by hand
    - { role: base_nginx_static_domain,
        vars: { my_server_name: 'testing-srvXX.staging.MYDOMAIN.TLD',
                my_ssl_prefix : '{{ _my_ssl_prefix }}',
                my_root_folder: '/usr/share/nginx/html3' } }

Expected Results

I expected 3 nginx servers to be created but only the last one is created. Executing a run with any of the 3 calls to base_nginx_static_domain present generates the correct single server configuration.

Actual Results

Only this part (the last one) is executed.

    # this creates another static server, root dir was created by hand
    - { role: base_nginx_static_domain,
        vars: { my_server_name: 'testing-srvXX.staging.MYDOMAIN.TLD',
                my_ssl_prefix : '{{ _my_ssl_prefix }}',
                my_root_folder: '/usr/share/nginx/html3' } }

[ghomem]

There is a possibly related issue ansible/ansible-documentation#90 but I want to stress that in this case the parameters are different for each of the role calls and the documentation is clear in saying:

"You have two options to force Ansible to run a role more than once.

• Passing different parameters [...]
• Using allow_duplicates: true"

https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#running-a-role-multiple-times-in-one-playbook

Is this a bug? I apologize if I misunderstood something.

[ghomem]

Someone with a similar issue:

https://stackoverflow.com/questions/62879235/how-to-reuse-ansible-role-with-different-set-of-variables-in-the-same-playbook

However, I didn't manage to make that solution work, nor it seems to be necessary according to the documents.

[sivel]

Please provide the details about the tasks that are being executed within base_nginx_static_domain

Including meta/main.yml as well as the contents of tasks/

[ghomem]

Hi,

It only has main.yml and the content is this

- name: Configure NGINX static server
  include_role:
    # FIXME the ansible galaxy module is broken
    #name: nginxinc.nginx_config
    # obtained from git
    name: ansible-role-nginx-config
  vars:
    nginx_config_start       : true # triggers an nginx reload on configuration change
    nginx_config_debug_output: false
    nginx_config_debug_tasks : false

    nginx_config_http_template_enable: true
    nginx_config_http_template:
      - template_file     : http/default.conf.j2
        conf_file_name    : '{{ my_server_name }}.conf'
        conf_file_location: /etc/nginx/conf.d/
        servers:
          - listen:
              - port: 443
                ssl : true
                opts: []
            server_name: '{{ my_server_name }}'
            ssl:
              cert                 : '/etc/ssl/certs/{{ my_ssl_prefix }}.nginx.bundle.crt'
              key                  : '/etc/ssl/private/{{ my_ssl_prefix }}.key'
              protocols            : '{{ NGINX_PROTOCOLS }}'
              prefer_server_ciphers: true
              ciphers              : '{{ NGINX_CIPHERS   }}'
              stapling             : true
              stapling_verify      : true
            autoindex : false
            locations :
              - location: /
                root    : '{{ my_root_folder }}'

[sivel]

That is your problem. The role included by include_role does not share the same variable scoping as the role that included it.

The variable precedence docs are being updated to explain this in more detail: https://github.com/ansible/ansible/pull/73939

Effectively, there is no special variable scoping for a parent-child relationship with a role that includes another. The included role is in the scope of the play, and does not have the same variable scoping as the role that called it.

A task within a role prefers it's own vars, but the tasks from an include_role does not as they are not part of the "root" role.

[ghomem]

Thank you for your answer. Not sure if I understand 100% :-)

The variables are passed without problems to the role base_nginx_static_domain and from that one to ansible-role-nginx-config. The problem is that at the playbook level ansible is chosing to only execute the last instance... which however takes the correct variables inside.

Anyway, the most important part is: how can I run the role base_nginx_static_domain multiples times from a server playbook in such a way that it internally can call the ansible-role-nginx-config ?

From what I have seen a role that runs inside a role should be included with include_role.

[sivel]

These bullet points explain it:

   #. role vars from all roles (defined in ``roles/*/vars/main.yml`` or ``vars:``)
   #. role dependency vars for tasks from a role (defined in ``roles/{dependent_roles}/vars/main.yml``)
   #. role vars for tasks from a role (defined in ``role/vars/main.yml``) [5]_ [8]_ [9]_

Only that first one applies to the tasks of ansible-role-nginx-config, because the tasks of ansible-role-nginx-config are not part of base_nginx_static_domain.

.. [5] Tasks in each role see their own role's vars if specified, or those from the role's dependencies. Tasks defined outside of a role see the last role's vars.

The tasks from ansible-role-nginx-config are defined outside of base_nginx_static_domain, and thus it uses the merged vars from all roles, which means it only ever sees the vars from the last defined use of base_nginx_static_domain

[ghomem]

Thank you, I understand better now.

Does this mean that it is not possible to execute multiple times a role that includes another role?

(I am trying to save on code duplication with this effort).

[sivel]

You can work around this, by changing how you define the vars, and instead use role params:

    # this creates a static server
    - { role: base_nginx_static_domain,
        my_server_name: 'testing-srv.staging.MYDOMAIN.TLD',
        my_ssl_prefix : '{{ _my_ssl_prefix }}',
        my_root_folder: '/usr/share/nginx/html' }

instead of:

    # this creates a static server
    - { role: base_nginx_static_domain,
        vars: { my_server_name: 'testing-srv.staging.MYDOMAIN.TLD',
              my_ssl_prefix : '{{ _my_ssl_prefix }}',
              my_root_folder: '/usr/share/nginx/html' } }

[ghomem]

That construct seems to work. Thank you.

But perhaps the documents are not clear enough? I read the docs in regards to role reuse and I missed this very important option to pass the parameters. Actually the document calls parameters what is actually used as "vars" and the parameter construction you mentioned above is not explained.

[sivel]

You are correct that our docs can be better here. There are very few people who understand it well enough to document it closely enough to accuracy. Unfortunately, not all of those docs were contributed by those few.

[ghomem]

OK, not sure if this issue should be closed or assigned to the documents. From my side: thank you for the helpful comments.

[sivel]

It should remain open to ensure we address the title of that section of docs at a minimum.

[ghomem]

Related:

https://medium.com/opsops/role-parameters-in-ansible-946386f32e77

[s-hertel]

I looked at this documentation recently and found it unclear too. I'm not sure if this is entirely related (old-style role usage vs includes), but both pertain to role signature and might be worth noting if this section of the docs is going to be rewritten. https://github.com/ansible/ansible/issues/56046#issuecomment-802415944 It would be great to have a complete list of things that can make up a role signature. It was on my to-do list, but I hadn't gotten to it.

[sivel]

I'm going to try to rewrite that documentation linked in the original report as well. parameters and variables (and their short forms) are often incorrectly used in the wrong context. If we were to assume that the role foo only contained something like:

- debug:
    msg: '{{ message }}'

Then it works as expected.

But because roles do not share a parent child relationship, the tasks from an included role, do not share the same precedence/scope of vars on the top level role. Only direct tasks of a role "prefer" their own vars. Tasks from other roles, see the merged scope of all roles.

So an include_role from another role, operates the same as of the include_role happened from outside of the top level role. Just saying that makes me dizzy...

[ghomem]

I admit that the verbal explanation about variable scoping in roles makes me dizzy as well but seeing that @sivel is using his time on this matter, I should perhaps support a little better the example I gave for the initial report, claiming that it actually represents a very logical use case.

1) for any popular Configuration Management framework we have builtin types and community modules of generic use (wide scope) 2) in order to avoid code multiplication it is often convenient to wrap those types, or resources from community modules, in more specialized (narrower scope) re-usable constructs (called roles or classes) 3) we then want to apply those constructs on the configuration of our systems, one or more times per system

With this we can achieve specialized, elegant and organized code.

In the example I used on this report:

1) ansible-role-nginx-config is a wide scope, complex community role 2) base_nginx_static_domain is a wrapper for a specific use of such role

and

3)

 - { role: base_nginx_static_domain,
        my_server_name: 'testing-srv.staging.MYDOMAIN.TLD',
        my_ssl_prefix : '{{ _my_ssl_prefix }}',
        my_root_folder: '/usr/share/nginx/html' }

is the instantiation of base_nginx_static_domain on a specific system (on the system playbook), instantiation which can occur a single time or multiple times.

Concrete need this addresses: we might want to create several nginx static servers with common internal configuration but variable server_name, ssl_prefix for the cert and key file names and root folder for serving files.

For comparison, if this was to be done with Puppet instead we would have the following:

1) community module puppet-nginx, which contains resources like nginx::resource::server and nginx::resource::location 2) our own base_nginx_static_domain wrapper that implements an nginx static server with common internal configuration but variable server_name, ssl_prefix for the cert and key file names and root folder 3) node declaration that instantiates base_nginx_static_domain one or more times for the system to be configured

It this case the wrapper mentioned above would be

a) a class if it should only be instantiated once per node b) a defined-type if it can be instantiated multiple times

Therefore, the scenario I provided as an example on this issue is what would be implemented in Puppet with a defined type base_nginx_static_domain. The defined type would receive server_name, ssl_prefix and root_folder as parameters and pass them to the underlying module.

Whatever the Configuration Management framework is I think this 3 layer model applies because:

i) community resources are always generic.. specialized use is left for the implementer ii) we want to avoid complexity on the declaration of systems because it leads to multiplication of code

So, unless we are operating very simple systems, we will need something in between and that something needs to support multiple instances.

I hope I made the case more clear. Sorry for the huge text and thanks for the support :-)

Reference:

https://puppet.com/docs/puppet/5.5/lang_defined_types.html

[nitzmahone]

This should be fixed in newer releases, feel free to reopen if it's still an issue.