jenkins oauth redirect fails with k8s module definition due json quoting

[ikke-t]

SUMMARY

I'm doing GitOps with Ansilbe k8s module. I setup jenkins, but k8s definition for jenkins oauth route fails to be set due ansible adding extra " around the json. Jenkins is so picky, that it will fail. I find no way to remove the extra " around json in k8s definition, other than doing a template and doing it in jinja. Which is ugly, as none of the other files are not templated.

How to escape internal json properly, or avoid ansible adding the quotes around json to avoid failure?

ISSUE TYPE

• Bug Report

COMPONENT NAME

k8s

ANSIBLE VERSION

rhel8

▶ ansible --version
ansible 2.9.14
  config file = /home/itengval/src/fevermap/ocp/ansible/ansible.cfg
  configured module search path = ['/home/itengval/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.8 (default, Dec  5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

CONFIGURATION

COLOR_DEBUG(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = gray
COLOR_DIFF_REMOVE(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = bright red
COLOR_ERROR(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = bright yellow
COLOR_UNREACHABLE(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = bright yellow
COLOR_VERBOSE(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = light blue
DEFAULT_CALLBACK_WHITELIST(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = ['profile_tasks']
DEFAULT_VAULT_PASSWORD_FILE(/home/itengval/src/fevermap/ocp/ansible/ansible.cfg) = /home/itengval/src/fevermap/ocp/ansible/.vault-pw

OS / ENVIRONMENT

fevermap/ocp/ansible  fix-jenkins-oauth ✗
▶ cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.2 (Ootpa)

fevermap/ocp/ansible  fix-jenkins-oauth ✗  
▶ rpm -q ansible
ansible-2.9.14-1.el8ae.noarch

fevermap/ocp/ansible  fix-jenkins-oauth ✗

▶ pip3 show openshift
Name: openshift
Version: 0.11.2
Summary: OpenShift python client
Home-page: https://github.com/openshift/openshift-restclient-python
Author: OpenShift
Author-email: UNKNOWN
License: Apache License Version 2.0
Location: /usr/lib/python3.6/site-packages
Requires: 

STEPS TO REPRODUCE

The problematic SA definition is here. I had to move away from specifying json there directly since ansible parser gets confused if having variables in middle of json. So I moved the definition into var, and instead use json filter to palce it into definiton.

https://gitlab.com/fevermap/fevermap/-/blob/master/ocp/ansible/roles/pipelines/tasks/main.yml#L20

- name: Jenkins ServiceAccount
  vars:
    oauth_redirect:
      kind: OAuthRedirectReference
      apiVersion: v1
      reference:
        kind: Route
        name: "{{ app }}-jenkins"
  k8s:
    state: "{{ state }}"
    definition:
      - apiVersion: v1
        kind: ServiceAccount
        metadata:
          annotations:
            serviceaccounts.openshift.io/oauth-redirectreference.jenkins: >-
              "{{ oauth_redirect|to_json }}"

The lines should end up with:

  annotations:
    serviceaccounts.openshift.io/oauth-redirectreference.jenkins: '{"kind": "OAuthRedirectReference",
      "apiVersion": "v1", "reference": {"kind": "Route", "name": "fevermap-jenkins"}}'

Instead, it is like this (not the extra " wrapping):

  annotations:
    "serviceaccounts.openshift.io/oauth-redirectreference.jenkins: '{"kind": "OAuthRedirectReference",
      "apiVersion": "v1", "reference": {"kind": "Route", "name": "fevermap-jenkins"}}'"

Instead, there is no way to have it without " or ' around it, which ruins the jenkins conf. Thus, it's not possible to end up with working config.

Jenkins redirect guide has examples: https://docs.openshift.com/container-platform/4.6/authentication/using-service-accounts-as-oauth-client.html

I run the fevermap playbook https://gitlab.com/fevermap/fevermap/-/blob/master/ocp/ansible/fevermap.yml

like this:

K8S_AUTH_VERIFY_SSL=false ansible-playbook -i konttikoulu_staging -e jenkins_source_repository_ref=fix-jenkins-oauth -e jenkins_source_repository_url='https://gitlab.com/ikkeT/fevermap.git' -e api_key='DDrn6Vp...' -e push_api_firebase_account='aGVpCg==' -e state=present  fevermap.yml

EXPECTED RESULTS

Able to login to jenkins.

ACTUAL RESULTS

Jenkins redirect does not work, and login fails.

K8S_AUTH_VERIFY_SSL=false ansible-playbook -i konttikoulu_staging -e jenkins_source_repository_ref=fix-jenkins-oauth -e jenkins_source_repository_url='https://gitlab.com/ikkeT/fevermap.git' -e api_key='DDrn6Vpq4nxe...' -e push_api_firebase_account='aGVpCg==' -e state=present  fevermap.yml

[tima]

@ikke-t A couple of us have tried to reproduce this quoting of the entire line problem and cannot do it. I'm not using RHEL8 and I'm running python 3.8, but I used the same versions of Ansible and the openshift client (fresh install in virtualenv).

Have you tried reproducing this using different versions of ansible or python or OSs? Is it only to_json on that one specific annotation?

[ikke-t]

odd, now I made a simplified reproducal for you guys, and tested again before sending, and it works. However, I updated my rhel8 ansible, and ocp clustere from 4.5 to 4.6. I will re-test the fevermap one too.

[ikke-t]

od. Now it works with fevermap too. I wonder how this is possible, I've put so many hours into figuring this out. Something behaves diferently after the updates. Well, I'll close this one. Sorry for the trouble, and thanks for looking into it.