remove_keys filter remove empty string from list

[hguermeur]

SUMMARY

Hi, I'm not sure if it is the expected behavior: remove_keys applied on [ "" ] always return [ ] in place of [ "" ]

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible.utils.remove_keys

ANSIBLE VERSION

ansible [core 2.14.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/ubuntu/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True

Same problem with:

ansible-playbook [core 2.12.3.post0]
  config file = None
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
  ansible collection location = /runner/requirements_collections:/home/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.8.12 (default, Sep 21 2021, 00:10:52) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 2.10.3
  libyaml = True

COLLECTION VERSION

# /home/ubuntu/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
ansible.utils     2.9.0  
community.general 6.5.0  
kubernetes.core   2.4.0  

CONFIGURATION

CONFIG_FILE() = /etc/ansible/ansible.cfg

OS / ENVIRONMENT

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy

Not environment dependent.

STEPS TO REPRODUCE

File test.yaml:

apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    annotations:
      rbac.authorization.kubernetes.io/autoupdate: "true"
    creationTimestamp: "2023-03-08T10:06:46Z"
    labels:
      kubernetes.io/bootstrapping: rbac-defaults
    name: extension-apiserver-authentication-reader
    namespace: kube-system
    resourceVersion: "184"
    uid: 99df70e6-6d4f-442a-8b95-bc4b2138b589
  rules:
  - apiGroups:
    - ""
    resourceNames:
    - extension-apiserver-authentication
    resources:
    - configmaps
    verbs:
    - get
    - list
    - watch
kind: List
metadata:
  resourceVersion: ""

Playbook to run:

---
- name: playbook test remove_keys
  hosts: localhost
  tasks:
    - name: debug file content without remove_keys
      ansible.builtin.debug:
        msg: "{{ lookup('file','test.yaml') | from_yaml }}"

    - name: debug file content with remove_keys
      ansible.builtin.debug:
        msg: "{{ lookup('file','test.yaml') | from_yaml | ansible.utils.remove_keys(target=['creationTimestamp','resourceVersion']) }}"

EXPECTED RESULTS

remove_keys should not remove the empty string from the apiGroups list:

TASK [debug file content with remove_keys] **************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": {
        "apiVersion": "v1",
        "items": [
            {
                "apiVersion": "rbac.authorization.k8s.io/v1",
                "kind": "Role",
                "metadata": {
                    "annotations": {
                        "rbac.authorization.kubernetes.io/autoupdate": "true"
                    },
                    "labels": {
                        "kubernetes.io/bootstrapping": "rbac-defaults"
                    },
                    "name": "extension-apiserver-authentication-reader",
                    "namespace": "kube-system",
                    "uid": "99df70e6-6d4f-442a-8b95-bc4b2138b589"
                },
                "rules": [
                    {
                        "apiGroups": [
                            ""
                        ],
                        "resourceNames": [
                            "extension-apiserver-authentication"
                        ],
                        "resources": [
                            "configmaps"
                        ],
                        "verbs": [
                            "get",
                            "list",
                            "watch"
                        ]
                    }
                ]
            }
        ],
        "kind": "List",
        "metadata": {}
    }
}

ACTUAL RESULTS

remove_keys remove the empty string from the apiGroups list (the ouput from the filter from_yaml is OK). Same result with matching_parameter=regex

TASK [debug file content with remove_keys] **************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": {
        "apiVersion": "v1",
        "items": [
            {
                "apiVersion": "rbac.authorization.k8s.io/v1",
                "kind": "Role",
                "metadata": {
                    "annotations": {
                        "rbac.authorization.kubernetes.io/autoupdate": "true"
                    },
                    "labels": {
                        "kubernetes.io/bootstrapping": "rbac-defaults"
                    },
                    "name": "extension-apiserver-authentication-reader",
                    "namespace": "kube-system",
                    "uid": "99df70e6-6d4f-442a-8b95-bc4b2138b589"
                },
                "rules": [
                    {
                        "apiGroups": [],
                        "resourceNames": [
                            "extension-apiserver-authentication"
                        ],
                        "resources": [
                            "configmaps"
                        ],
                        "verbs": [
                            "get",
                            "list",
                            "watch"
                        ]
                    }
                ]
            }
        ],
        "kind": "List",
        "metadata": {}
    }
}

[hguermeur]

Hi, Do you succeed to reproduce the issue ?

Thks, Hervé

[vbotka]

I was able to reproduce the issue

  t: [k0, k1]
  l1:
    - {k0: A, k1: B, k2: [""], k3: foo}
    - {k0: C, k1: D, k2: [""], k3: bar}
  r1: "{{ l1 | ansible.utils.remove_keys(target=t) }}"

gives

  r1:
  - k2: []
    k3: foo
  - k2: []
    k3: bar