Double quotes removed from strings

[uroshercog]

Describe the bug Running kustomize removes double and/or single quotes from strings.

If a value in a config map is a template placeholder and the actual application value is not yet known because the built resources are post-processed by another process, the quotes should not be removed. Removing them can lead to producing invalid config maps. For example if the value of the placeholder is a commit SHA which just happens to be all numeric then the value in the config map is interpreted as a number by Kubernetes and rejected with a validation error. Values in config maps should always be quoted given that they can only be strings anyways.

Files that can reproduce the issue

config-map.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: example-config-map
data:
  key: "${placeholder}"

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: example
resources:
  - config-map.yaml

Expected output

apiVersion: v1
kind: ConfigMap
metadata:
  name: example-config-map
data:
  key: "${placeholder}"

Actual output

apiVersion: v1
kind: ConfigMap
metadata:
  name: example-config-map
data:
  key: ${placeholder}

Kustomize version

{Version:kustomize/v4.5.7 GitCommit:56d82a8378dfc8dc3b3b1085e5a6e67b82966bd7 BuildDate:2022-08-02T16:35:54Z GoOs:darwin GoArch:arm64}

Platform

MacOS M1 Monterey 12.6

[KnVerey]

This is highly related to https://github.com/kubernetes-sigs/kustomize/issues/4146, https://github.com/kubernetes-sigs/kustomize/issues/3412 and others where the root cause is a possibly-non-string value in what Kustomize theoretically could recognize as a string field. I believe the conclusion so far has been that the quotes are being stripped as superfluous by the YAML library we are using, so although we agree with the behaviour being undesirable, the universal fix isn't obvious or simple (e.g. should we try to force the style to be preserved, and how? or should we use schema knowledge to override after the fact, and where?).

/triage accepted

[emmaLP]

@KnVerey Any update on this please?

We have just hit this issue on the same version when reading config from an env file and then the value is then used to set some labels.

Using kustomize v4.5.7

If we try wrap the value in double or single quotes to preserve the value as a string, we then get validation errors such as

Deployment.apps "service-name" is invalid: [metadata.labels: Invalid value: "'9079318'": a valid label must be an empty str
ing or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?'), spec.template.labels: Invalid value: "'9079318'": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')]  

Update 2023-03-14: Tried the same config on the following versions and still experiencing the same behaviour:

1. v5.0.0
2. v4.5.5
3. v4.0.5

[gandarez]

Facing the same problem when setting loadBalancerIP for traefik which expects an IP address surrounded by double quotes.

[gandarez]

Any update on this?

[CpuID]

Also looks related to https://github.com/kubernetes-sigs/kustomize/issues/3485 - which was closed due to vars deprecation, but the quotes issue looks identical.

[snukone]

This is still an issue. Tried kustomize build with kustomize version 5.0.0:

configMapGenerator:
- behavior: merge
  literals:
  - ENVIRONMENT='${TEST}'
name: config

Turns into after kustomize build:

apiVersion: v1
data:
  ENVIRONMENT: ${TEST}

After that i do an envsubst and replace ${TEST} with 12345. The resulting YAML isnt possible to apply with kubectl due to conversion error: cannot convert int64 to string (because of ENVIRONMENT: 12345). Escaping the value 12345 with multiple single quotes or backslahes makes kubectl apply work, but the multiple single quotes or backslash arent replaced (e.g. lead to ENVIRONMENT: \'12345\' in the new configmap)

[snukone]

Solution: Using block scalar style as mentioned here with Kustomize v5.0.0 works: https://github.com/kubernetes-sigs/kustomize/issues/4233#issuecomment-1009278304 https://stackoverflow.com/a/21699210/7746963

configMapGenerator:
- behavior: merge
  literals:
  - |
    ENVIRONMENT=${TEST}
name: config

Now building the Kustomization, subsitute ${TEST} with 12345 and applying it with kubectl results in the following ConfigMap:

apiVersion: v1
data:
ENVIRONMENT: |
    12345

You will have a linebreak at the end of the value (>- or |- doesnt help), but when deploying the application and checking the environment variables within the container of the pod with env the value is without linebreak and the application works as designed:

$ env | grep ENV
ENVIRONMENT=12345
ENVPATH=/dev

[alxndr42]

The same problem exists in replacements. I need to take a string value and put it in an annotation, which fails because the value gets converted to an int.

[Pradumn99]

Do we have any solution to above issue as I still observe this behaviour of stripping away the double quotes ?

[krokofant]

Replacements seems conventionally useless if it breaks quoted numbers 😕

[pastukhov]

Any news?

[NGPixel]

It's quite annoying that the maintainers don't seem interested in addressing this issue. Dynamically replacing vars in the kustomize output, outside of the "kustomize process", is a totally valid workflow.

A workaround is to use yq (the YAML equivalent to jq) to add back double quotes to values that are environment variables:

kubectl kustomize foobar > manifest.yaml
yq -i --unwrapScalar=false '(.. |= select(. == "$*") style="double")' manifest.yaml

You can then replace vars further down the pipeline, e.g. with envsubst:

kubectl kustomize foobar > manifest.yaml
yq -i --unwrapScalar=false '(.. |= select(. == "$*") style="double")' manifest.yaml
envsubst < manifest.yaml | kubectl apply -f -

or as a single pipeline:

kubectl kustomize foobar | yq --unwrapScalar=false '(.. |= select(. == "$*") style="double")' | envsubst | kubectl apply -f -