Recommended way to pass Template parameters when running playbook

[donovanmuller]

What is the recommended way of passing Template params when running the playbook? I.e. at playbook execution time.

Currently you can specify Template parameters in the ...-params file pointed to by the params property in inventory/group_vars/all.yml. So using the example below:

---
openshift_cluster_content:
- object: projectrequest
  content:
  - name: test
    template: "{{ inventory_dir }}/../files/projects/test-template.yml"
    params: "{{ inventory_dir }}/../files/projects/test-params"

{{ inventory_dir }}/../files/projects/test-params can contain my Template parameters to be used when processing the Template.

How could you allow for specifying parameters when running the playbook? I.e. when running from the command line or even via AWX/Tower, how could you add any required parameters or even replace existing parameters in {{ inventory_dir }}/../files/projects/test-params?

[oybed]

@donovanmuller The openshift-applier does not support supplying / passing individual parameters, or a runtime override of param files, as this isn't necessarily the use case it is meant for. The primary purpose for the openshift-applier is to apply a collection of templates with a corresponding collection of parameters, and do so in a controlled fashion - i.e.: think Infrastructure-as-Code (IaC). This typically means supplying a lot more than just a few parameters and most often not maintained as runtime values (as that goes against the purpose of tracking the content).

Having said that; we do support tweaking some existing parameters at runtime, through the use of the make-applier-projects-unique role, but this is mainly meant for PoC or testing purposes where you want to ensure the namespaces don't clash.

We are, however, interested in input on these and curious to know more about the needs you have around runtime parameters. Can you please elaborate?

[donovanmuller]

@oybed thanks for the reply.

In my case I have a collection of templates and resources that make up a specific environment. I would like to duplicate this environment in development, staging etc. projects by using the same templates and resources but being able to change certain environment specific parameters (like namespace name etc.) via variables.

This allows me to use the same version controlled inventory files for multiple projects/environments without duplication. In my case I would like to run my openshift-applier inventory files from AWX/Tower and having the ability to specify parameters via Job Template variables gives me an easy way to create multiple environments by simply copying a Job Template and changing the variables.

I have came up with a rough implementation that allows me to do the following. Given a Project template as follows:

apiVersion: v1
kind: Template
metadata:
  name: team-devops-template
  annotations:
    openshift.io/display-name: Team DevOps project
labels:
  template: team-devops-template
objects:
  - apiVersion: v1
    kind: ProjectRequest
    apiVersion: v1
    metadata:
      name: ${TEAM_NAME}
    displayName: ${TEAM_DISPLAY_NAME}
    description: DevOps environment for ${TEAM_DISPLAY_NAME}
parameters:              
  - name: TEAM_NAME
    displayName: Team name
    required: true      
  - name: TEAM_DISPLAY_NAME
    displayName: Human readable team name
    required: true

and the following openshift_cluster_content:

---
openshift_cluster_content:
- object: projectrequest
  content:
  - name: team-devops
    template: "{{ inventory_dir }}/../files/projects/team-devops.yml"
    params: "files/projects/params"
    extra_params:
      TEAM_NAME: "{{ team_name }}-devops"
      TEAM_DISPLAY_NAME: "{{ team_display_name }} DevOps"

extra_params is something I added (for lack of a better name) that allows me to use variables for the parameter values.

I can then apply this with the following:

$ ansible-playbook -i inventory/hosts site.yml \
  --connection=local \
  --extra-vars \
    "team_name=test \
    team_display_name='Test project'"

I can create a brand new project with the same resources but different name etc. by simply changing the extra-vars values.

It also supports overriding existing parameter values specified in the params file.

[springdo]

@donovanmuller - this would be an awesome feature of the applier. If we could pass in the extra_params (or whatever it's called) like that you could reduce the amount of duplication across the ansible part of the applier and the what goes in the cluster

[sherl0cks]

@oybed the big use case where this sort of thing would be handy is namespace vars, particularly when you reference a namespace for a rolebinding (e.g. the Jenkins rolebinding). in practice, using a role to modify these params in params files on disk hasn't really been the catch all solution we were hoping for, so I'm supportive of this type of feature.

[etsauer]

@donovanmuller I think there is one feature in place already that may addres your need.

This is the idea of parameter directories.

---
openshift_cluster_content:
- object: projectrequest
  content:
  - name: team-devops
    template: "{{ inventory_dir }}/../files/projects/team-devops.yml"
    params: "files/projects/params/project-defs/"

$ ls files/projects/params/project-defs/
teamA teamB
$ cat files/projects/params/project-defs/teamA
TEAM_NAME=teamA-devops
TEAM_DISPLAY_NAME=Team A DevOps
$ cat files/projects/params/project-defs/teamB
TEAM_NAME=teamB-devops
TEAM_DISPLAY_NAME=Team B DevOps

I realize that we're missing documentation for this feature. so I've opened #17 for this.

[sherl0cks]

@etsauer that doesn't fit the need IMHO. I want to pass in the param via a normal ansible variable. Right now we are stuck with all params being in a file, and that can be a pain in some cases, particularly with namespaces.

[donovanmuller]

@etsauer thanks for the example around the use of params like that. Could come in handy. However, as @sherl0cks mentioned that wouldn't best suit my use case.

Keep in mind, my use case is also strongly influenced by the need to run the same applier based playbooks from AWX/Ansible Tower. I.e. I want to be able to create and then copy, as needed, Job Templates for creating resources but only needing to change specific variables for each (copied) Job Template. That way I have a cookie cutter approach to creating non trivial resource collections from AWX/Tower. See screenshots below...

My patched openshift-applier currently works quite well with the above use case in mind. Would it be useful to share as a PR?

Screenshots:

Given this Job Template referencing an openshift-applier based repo:

I can copy this Job Template and change the EXTRA VARIABLES accordingly:

[springdo]

@etsauer, I agree with @sherl0cks. The functionality I was thinking about is a using ansible variables because there is so much repetition in using them when running a local .openshift-applier inventory. So for example imagine I have a group vars that includes

---
openshift_cluster_content:
- object: app-builds
  content:
  - name: todolist-api-build
    template: "{{ playbook_dir }}/templates/todolist-api-deploy.yml"
    params: "{{ playbook_dir }}/params/dev"
    namespace: "{{ ci_cd_namespace }}"
    tags:
    - build
- object: deploy-dev
  content:
  - name:  todolist-api
    template: "{{ playbook_dir }}/templates/todolist-api-deploy.yml"
    params: "{{ playbook_dir }}/params/dev"
    namespace: "{{ dev_namespace }}"
    tags:
    - deploy
    - dev
  - name:  todolist-api-db
    template: "{{ playbook_dir }}/templates/mongodb.yml"
    params: "{{ playbook_dir }}/params/mongodb"
    namespace: "{{ dev_namespace }}"
    tags:
    - deploy
    - dev

and an apply.yml with

---
- name: Build and Deploy todolist-api 
  hosts: app
  vars:
    ci_cd_namespace: donal-ci-cd
    dev_namespace: donal-dev
    test_namespace: donal-test
  tasks:

Now I add some params for dev, test, build etc I end up with repetition of variables (NAME & NAMESPACE for example) all over the place. This ovs would get more complex if there were more than 2 environment too.

in dev:
PIPELINES_NAMESPACE=donal-ci-cd
NAME=todolist-api
NAMESPACE=donal-dev

in test:
PIPELINES_NAMESPACE=donal-ci-cd
NAME=todolist-api
NAMESPACE=donal-test

Having the vars defined in the apply.yaml and let them be inserted would mean less re-typing of stuff across the template params & ansible code

[sherl0cks]

@donovanmuller I for one would like to see the PR - even if it is just for discussion purposes. In general, there is a design principle here to keep the applier small and simple, so the notion of really adding any more features than we have now will be scrutinized pretty hard. That said, this feels to me like a good candidate.

[donovanmuller]

@sherl0cks ok, I'll try clean it up and push over the weekend. 👍