Add extra_vars to every job event sent over `job_events`

[brsanche]

ISSUE TYPE

• Feature Idea

SUMMARY

This issue is related to the following:

RFE: Add extra_vars for callbacks to the log for external logger like Splunk

User wants to use information that exist in extra_vars from a job during the processing of job_events in Splunk. It would be easier for the user if the extra_vars field and value(s) were included in every event that goes over job_events.

[chrismeyersfsu]

Let me see if I understand the ask here. When you run the below playbook, today, there is no way, from the ansible output alone, for you to tell that the variable inventory_hostname was used to generate the output echo localhost. What you want is variable province. Is that right?

- hosts: localhost
  gather_facts: false
  tasks:
    - shell: "echo {{ inventory_hostname }}"

➜  /tmp ansible-playbook main.yml -vvv
ansible-playbook [core 2.11.5]
  config file = None
  configured module search path = ['/home/meyers/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/meyers/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.9.7 (default, Aug 30 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]
  jinja version = 2.11.3
  libyaml = True
No config file found; using defaults
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: main.yml *********************************************************************************************************************************************************************************************************************************
1 plays in main.yml

PLAY [localhost] ***********************************************************************************************************************************************************************************************************************************
META: ran handlers

TASK [shell] ***************************************************************************************************************************************************************************************************************************************
task path: /tmp/main.yml:4
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: meyers
<127.0.0.1> EXEC /bin/sh -c 'echo ~meyers && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/meyers/.ansible/tmp `"&& mkdir "` echo /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659 `" && echo ansible-tmp-1636743145.9021337-1912690-110816152490659="` echo /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659 `" ) && sleep 0'
Using module file /usr/local/lib/python3.9/site-packages/ansible/modules/command.py
<127.0.0.1> PUT /home/meyers/.ansible/tmp/ansible-local-1912686v0isgci0/tmpsyzh0kan TO /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659/ /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/meyers/.ansible/tmp/ansible-tmp-1636743145.9021337-1912690-110816152490659/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true,
    "cmd": "echo localhost",
    "delta": "0:00:00.002306",
    "end": "2021-11-12 13:52:26.180437",
    "invocation": {
        "module_args": {
            "_raw_params": "echo localhost",
            "_uses_shell": true,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": false
        }
    },
    "msg": "",
    "rc": 0,
    "start": "2021-11-12 13:52:26.178131",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "localhost",
    "stdout_lines": [
        "localhost"
    ]
}

[chrismeyersfsu]

This can be tricky from a performance point of view. There is limitation on the payload size we can send from AWX to rsyslog due to using unix domain sockets. The addition of extra_vars in every job_event would need to take this into consideration.

[yohan-b]

I work for the user (Red Hat Ansible Tower customer) who is asking for this. The need is to be able to filter job events logs efficiently with Splunk when a generic template/playbook is used (with the role or script to launch passed as an extra var instead of creating hundreds of identical template jobs+playbook just to get a different name to filter on). The first idea was to add extra_vars to every job events, but it would work just as well to be able to add a short string label (prompt on launch) when launching a job template. This should not cause a performance issue.