Create ConfigMap from a directory containing configuration files

[tashoyan]

SUMMARY

Our application has all configuration files in a directory conf. When deploying to Kubernetes, we used to create a ConfigMap from the entire directory:

kubectl create configmap app-conf --from-file=conf

With Ansible k8s module it is impossible for now. It is especially bad, because now we deploy applications from a Docker container running in a Jenkins pipeline. This Docker container has Ansible inside, but it is not supposed to have also kubectl.

Would be good to make it like this:

- name: Create app config
  k8s:
    kind: ConfigMap
    name: app-conf
    src: <path to conf dir>

ISSUE TYPE

• Feature Idea

COMPONENT NAME

Ansible k8s module

ADDITIONAL INFORMATION

Ansible 2.10.3 Python 3.7.3

Currently we are using a workaround:

1. A shell script reads all the files in the conf directory and generates the YAML content as expected by kubectl apply -f descriptor.yaml.
2. Then we feed this auto-generated YAML descriptor as src argument to k8s module.

There is a similar issue https://github.com/ansible/ansible/issues/55329, but it does not provide a solution.

[tima]

I'm not opposed to this idea, but I'd have an issue with adding a param and functionality for a specific resource to the k8s module. That module is meant to be generic utility so something this specific wouldn't work. (Imagine adding all of the resource specific functions to this one module over time.)

Have you tried using the template param and building the configmap using a fileglob filter and a Jinja template? It's a bit more involved than the purpose made module, but it would avoid needing kubectl as you've mentioned.

[tashoyan]

My first attempt was "true Ansible": template + jinja + fileglob. The solution turned out to be cumbersome, unmaintainable and unreadable. That's why I resorted back to a helper shell script.

Is it necessary to add a new parameter? It might be possible to use the existing src parameter and pass the directory with configuration files as an argument.

- name: Create Tomcat configuration
  k8s:
    kind: ConfigMap
    name: app-conf
    namespace: prod
    src: tomcat/etc
    state: present

It would be great to make it in one Ansible step without workarounds.

[tima]

I agree about avoiding workarounds. My suggestion was to help with what you have to work with now.

I think we need to be careful we don't overload params with too much and resource specific logic. That could make things unwieldy and confusing. While src works for your specific example of this use case, we need to consider if there more to it. Will it be useful and work for another resource kind? What if a user has files in the directory that should not be read in? Do we need a complex data structure for that param or does that start to cross the line to where we should have a module for this specific purpose? Those are just what questions came to my mind.

This isn't a no as much as I'm trying to get us to think through this some more before we implement something.

[tashoyan]

Let me try to answer some questions.

Will it be useful and work for another resource kind? For Secrets it could work the same way as for ConfigMaps.

What if a user has files in the directory that should not be read in? Just include all files in the target ConfigMap. If user does not want some files, he does not put them in the src directory. For example, there might be a file with a SSL private key. User does not want to include the private key in the ConfigMap, so the user keeps this file in a separate directory (and possibly creates a Secret).

[Akasurde]

@tashoyan Could you please check if ansible-collections/community.kubernetes#350 works for you? I am still testing this.

$ cat k8s_config.yml

---
- hosts: localhost
  tasks:
    - name: create the configmap
      k8s:
        state: present
        namespace: default
        src: "./config_maps"

TASK [create the configmap] *******************************************************************
task path: /328/k8s_config.yml:4
changed: [localhost] => {"changed": true, "result": {"results": [{"changed": true, "method": "create", "result": {"apiVersion": "v1", "data": {"foo": "foo: bar\n"}, "kind": "ConfigMap", "metadata": {"creationTimestamp": "2021-01-22T06:26:05Z", "name": "foo-configmap-2", "namespace": "default", "resourceVersion": "7035758", "selfLink": "/api/v1/namespaces/default/configmaps/foo-configmap-2", "uid": "d04ac71c-dd1e-493c-bcdf-f9b67420e8d3"}}, "warnings": []}, {"changed": true, "method": "create", "result": {"apiVersion": "v1", "data": {"foo": "foo: bar\n"}, "kind": "ConfigMap", "metadata": {"creationTimestamp": "2021-01-22T06:26:05Z", "name": "foo-configmap-1", "namespace": "default", "resourceVersion": "7035759", "selfLink": "/api/v1/namespaces/default/configmaps/foo-configmap-1", "uid": "8c31ab6f-914d-473b-8ef1-95c365328fd5"}}, "warnings": []}]}}

# kubectl get configmaps
NAME                              DATA   AGE
foo-configmap-1                   1      1m
foo-configmap-2                   1      1m

[tashoyan]

Hi @Akasurde The PR ansible-collections/community.kubernetes#350 does not seem to address the issue.

Let's start with an example: we want to create a ConfigMap from a directory conf containing 3 configuration files:

conf/
  log4j.xml
  metrics.properties
  server.conf

We want exactly the same result as the following command does:

kubectl create configmap app-conf --from-file=conf

So the ConfigMap app-conf will have content like:

kubectl get configmaps app-conf -o yaml

apiVersion: v1
data:
  log4j.xml: |
    <?xml version="1.0" encoding="iso-8859-1"?>
    ...
  metrics.properties: |
    master.source.jvm.class=org.apache.spark.metrics.source.JvmSource
    ...
  server.conf: |
    tcp-port = 7180
    ...

Having this ConfigMap app-conf, we can mount it in a directory conf inside a pod and have the entire application configuration.

The idea of this issue is to enable in Ansible k8s module something like this:

- name: Create application configuration
  k8s:
    kind: ConfigMap
    name: app-conf
    namespace: prod
    src: conf
    state: present

And this task creates the desired ConfigMap.

What I see in the PR ansible-collections/community.kubernetes#350: read all YAML files from a directory and create Kubernetes objects for all YAML definitions. This is something different.

[Akasurde]

@tashoyan Thanks for information. Could you please try following -

---
- hosts: localhost
  tasks:
    - set_fact:
        my_data: "{{ mydata | default({}) | combine({ item | basename : lookup('file', item) }) }}"
      with_fileglob: "config/*"

    - name: Create app config
      community.kubernetes.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: ConfigMap
          metadata:
            name: game-demo
            namespace: default
          data: "{{ my_data }}"

# kubectl get configmaps game-demo -o yaml
apiVersion: v1
data:
  server.conf: |-
    tcp-port = 7180
    udp-port = 7182
kind: ConfigMap
metadata:
  creationTimestamp: "2021-02-02T13:13:55Z"
  name: game-demo
  namespace: default
  resourceVersion: "7489365"
  selfLink: /api/v1/namespaces/default/configmaps/game-demo
  uid: 730748cc-5d6b-46d4-9bee-a72343855d2b

[Akasurde]

@tashoyan Could you please confirm if the above solution works for you? Thanks.

needs_info

[tashoyan]

@Akasurde Yes, this workaround works for me

[Akasurde]

@tashoyan Thanks for the feedback.

[Akasurde]

@tashoyan @tima @geerlingguy @gravesm Do you think a lookup plugin will be sufficient to solve this requirement?

The lookup plugin will basically replace the set_fact task from the above workaround without user intervention.

    - name: Create app config
      community.kubernetes.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: ConfigMap
          metadata:
            name: game-demo
            namespace: default
          data: "{{ lookup('unamed_configmap_plugin', dir='config/*') }}"

[tashoyan]

In my opinion, using lookup plugin is a quite obscured way. If one wants to deal with Kubernetes, he tries the k8s plugin first. It's not obvious that some Kubernetes-specific functions are provided by lookup(...). Practically it looks same as the workaround with lookup(file) - no way to figure out the correct way without help from an Ansible contributor.

[tima]

I've been wondering if this use (reading a batch of files from a directory) could be worked into ansible-collections/kubernetes.core#19. I'm still of the mind we need to be careful here and refrain from adding too much and resource specific logic that could make things unwieldy and confusing. I understand why a lookup may not be ideal, but I'm hesitant to build too much for this specific use case into the module. We could do something simple now like read the entire directory regardless what it is, but I'm certain that full fileglob filtering support replicate the lookup('fileglob', ...) function would soon follow.

Not a "no" just a still thinking about what's the smartest way to implement this.