Generator does not honor kustomization.yaml namespace

[hmehta]

I noticed that KSOPS generator doesn't honor the namespace set in kustomization.yaml. See my example project for details:

Simple Kustomize project with one base, one overlay:

base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml

base/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: exampleservice
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: exampleservice
          volumeMounts:
            - name: configs
              mountPath: /configs
              readOnly: true
            - name: secrets
              mountPath: /run/secrets
              readOnly: true
      volumes:
        - name: configs
          configMap:
            name: exampleservice-config
        - name: secrets
          secret:
            secretName: exampleservice-secret

overlays/test/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
configMapGenerator:
  - name: exampleservice-config
    files:
      - example.conf=configs/example.conf.json
generators:
  - ksops-secret-generator.yaml

overlays/test/ksops-secret-generator.yaml:

apiVersion: viaduct.ai/v1
kind: ksops
metadata:
  name: exampleservice-ksops-generator
  annotations:
    config.kubernetes.io/function: |
      exec:
        path: ksops
secretFrom:
  - metadata:
      name: exampleservice-secret
      annotations:
        kustomize.config.k8s.io/needs-hash: "true"
    type: Opaque
    files:
      - example.creds=secrets/example.creds.enc.json

overlays/test/configs/example.conf.json:

{
  "This is": "an example"
}

overlays/test/secrets/example.creds.enc.json (encrypted with the sops test keys):

{
    "This is": "ENC[AES256_GCM,data:eMo6YInUjee639kUjWUH/bM=,iv:X5+lodJyp6ggIcEFeeoXcS2GpJstQwgflBOyNUTkPMM=,tag:+ofQtLQbmcXWFGkpSi89Zg==,type:str]",
    "sops": {
        "kms": null,
        "gcp_kms": null,
        "azure_kv": null,
        "hc_vault": null,
        "age": null,
        "lastmodified": "2024-04-08T07:13:27Z",
        "mac": "ENC[AES256_GCM,data:a2AR7IqSkaO+/uuE3U64DkoULyriNh0pOzIm4LJ+eG5HPKiq8NdyxQvkLuWqI9pVivqsm1zwWQDFNbBz6u1VTmNTxVfgdNECoNtxYEFnQQefyQNEaQ/zRiwlLdIVsylv+PSHUXbUivCeEhStmkhOOVKL30CQLoZKdEYv9TC7ZBo=,iv:+DLIlCBg8NaHRwhmcq1JziCswTYz/6l/TSbONKQu8qc=,tag:yngcg5bAazNLCo/h9KoNTw==,type:str]",
        "pgp": [
            {
                "created_at": "2024-04-08T07:13:27Z",
                "enc": "-----BEGIN PGP MESSAGE-----\n\nhQEMAyUpShfNkFB/AQgAqmDIlhRxxDfbwnvTPJgkUgQyYkykeeBnTdVcc+ptJpPd\ny8Ka2KZ9drGb2Nx4OCsOKAtHrgfdCRgzc4yX1E9/fPVj2Nv2zwALFHM279kIadCl\nQ4skWB9jj+bmCeel7mFYbNV4s6Hbv1N41/Hs0fzaYYda2+y4OglXdzSa9LWnB2nR\nhtwXiBMUedNNu5SWQug1EASCIhxl/xUfs9E4S5p85irz8bSWHC4PkAhk8uoLso8w\nj9FY3lLz0qhCxJSc14NIT5jO16o03iOI662k79kfn2HNqw67rlhsKcjdFebR/i6x\n8nA8kxuqwFVNbKlqU3Nho+h/Z1rKq16TvkZhSsIjxNJeARGNxcRVuCV7oo1ReTLV\ns28I7fXN5BEJAATqKKqSEfMmz10TX65gWbWdn0ezS1K09gtk2vGExPIJfhjOD1Fs\nzume4z8JlZxKk745esUGG5RQ7wTd+4Dc3TRqgGGgEg==\n=MZEX\n-----END PGP MESSAGE-----",
                "fp": "FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"
            }
        ],
        "unencrypted_suffix": "_unencrypted",
        "version": "3.8.1"
    }
}

Then building this using kustomize build --enable-alpha-plugins --enable-exec . I get what I expect:

apiVersion: v1
data:
  example.conf: |
    {
      "This is": "an example"
    }
kind: ConfigMap
metadata:
  name: exampleservice-config-k2bmd7kk7f
---
apiVersion: v1
kind: Secret
metadata:
  name: exampleservice-secret-c5gc6b7g5g
stringData:
  example.creds: "{\n\t\"This is\": \"very secret value\"\n}"
type: Opaque
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: exampleservice
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: exampleservice
        volumeMounts:
        - mountPath: /configs
          name: configs
          readOnly: true
        - mountPath: /run/secrets
          name: secrets
          readOnly: true
      volumes:
      - configMap:
          name: exampleservice-config-k2bmd7kk7f
        name: configs
      - name: secrets
        secret:
          secretName: exampleservice-secret-c5gc6b7g5g

The spec.template.spec.containers.volumes[1].secret.secretName is what is expected since the Secret is also created to the namespace that is selected in the current context. But now when I set the namespace for everything in base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: example-namespace
resources:
- deployment.yaml

Now building this results into situation where the Secret doesn't have a namespace and even the content hash in the name gets dropped due to targeting failing (added few notes as comments to point it out better):

apiVersion: v1
data:
  example.conf: |
    {
      "This is": "an example"
    }
kind: ConfigMap
metadata:
  name: exampleservice-config-k2bmd7kk7f
---
apiVersion: v1
kind: Secret
metadata:
  # NOTE: No namespace here, all other resources does have the namespace set in the base/kustomization.yaml
  name: exampleservice-secret-c5gc6b7g5g
stringData:
  example.creds: "{\n\t\"This is\": \"very secret value\"\n}"
type: Opaque
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: exampleservice
  namespace: example-namespace
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: exampleservice
        volumeMounts:
        - mountPath: /configs
          name: configs
          readOnly: true
        - mountPath: /run/secrets
          name: secrets
          readOnly: true
      volumes:
      - configMap:
          name: exampleservice-config
        name: configs
      - name: secrets
        secret:
          # NOTE: I would expect this to become exampleservice-secret-c5gc6b7g5g instead!
          secretName: exampleservice-secret

This issue can be worked around by setting the same namespace explicitly in the overlays/test/ksop-secret-generator.yaml .secretFrom[0].metadata.namespace, but that's a bit clumsy and works quite differently than the Kustomize provided ConfigMap and Secrets Generators.

I hope you could fix this, but if it is intended behaviour, then I think documenting this better in the section where you give examples on providing Secrets using the secretFrom would help.

[dosubot[bot]]

Hello @hmehta! I'm Dosu, your friendly neighborhood bot here to lend a hand with bugs, answer your questions, or guide you on your journey to becoming a contributor, all while you're waiting for one of our human maintainers to jump in. I just wanted to drop in and say hi, introduce myself, and let you know I'm diving into your issue to give you a full answer shortly. Hang tight!

[hmehta]

This also seems to work as expected if I set the namespace in the OVERLAY kustomization.yaml, so I am somehow leaning towards this actually being either expected behavior from Kustomize side, or an issue there. As I went through the code, the whole metadata is used in the generated Secret without any modifications, so it should contain the modified metadata with the content hashed name when targeting succeeds (name + namespace matches).

Feel free to close this ticket. Maybe this will at least generate useful history for someone else trying to resolve the same problem.

[dosubot[bot]]

Hello @hmehta,

Thank you for your detailed follow-up and for sharing your findings with the community. It's great to see you've identified a workaround by specifying the namespace in the overlay's kustomization.yaml and also considered the behavior when setting the namespace directly in the ksops-secret-generator.yaml.

Your observation about the namespace being correctly applied when specified in the overlay, and your insight into the handling of metadata in the generated Secret, are valuable. It seems that this behavior, while initially unexpected, can indeed be managed through the configurations you've mentioned.

Given your analysis and the workaround you've discovered, it appears this issue might be more related to how Kustomize handles namespace propagation and metadata processing in conjunction with KSOPS, rather than a direct issue with KSOPS itself. Your willingness to share this information will undoubtedly help others who might encounter similar situations.

If you believe this behavior should be more clearly documented, or if there are improvements to be made in how KSOPS handles such scenarios, I encourage you to share these suggestions. Your input could lead to enhancements that benefit the broader community.

For now, since you've found a solution that works for your case and suggested that this ticket can be closed, we'll proceed accordingly. However, should you have any more questions, insights, or if you encounter other challenges, please don't hesitate to reach out again.

Your contributions and proactive approach to solving this issue are greatly appreciated!

Best regards, Dosu

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.