configMapGenerator / "kubectl create configmap" not adding quotes to large (>64-bit) hex numbers.

renaudguerin commented 1 year ago

What happened?

Hi,

I'm using configMapgenerator with a .env file that contains a variable whose value is a 128-bit number in hex notation.

I've discovered that configMapGenerator (and also kubectl create configmap) processes 64-bit and larger than 64-bit hex numbers differently, adding quotes to the generated output in the first case and not in the second.

I would guess the root cause is that <=64-bit hex strings get parsed as an int64, but larger numbers remain uninterpreted strings. However, that is not true for the decimal representation of the same 64-bit and 128-bit numbers I tested in hex : in decimal they both appear to be parsed as numbers, and both get double quotes in the output ! More inconsistency here ...

For context, I found this while debugging an issue with our manifest linter (Kubevious) : it balks at the 128-bit hex value in the ConfigMap because it expects a string but does parse it as a number (whereas Kubernetes doesn't).

What did you expect to happen?

I believe the current behaviour is confusing and Kustomize should treat all numeric values consistently. I also can't think of a great workaround right now :

If I add double quotes around the source value, configMapgenerator will add its own single quotes to the output, and that might break the application using it (via an env var in the Deployment loaded from CM with envFrom)
If I leave the source hex value unquoted, and (if and only if) it's >64-bit, the resulting CM value will also be unquoted, which will break the Kubevious linter.
I can't see a way to end up with the hex number inside double quotes in the ConfigMap (which according to @KnVerey here seems to be the desired behavior). As I've just explained, I can get either no quotes or nested quotes.

I can see there has been a lot of discussion already around quotes (at least #5124, #4845 , #3440, #3412, #4570, #4525), hopefully this brings another useful data point.

How can we reproduce it (as minimally and precisely as possible)?

# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
  - name: test-config
    envs:
      - test.env

# test.env
# 2^64-1
A=0x000000000000000000000000FFFFFFFFFFFFFFFF
AA="0x000000000000000000000000FFFFFFFFFFFFFFFF"
# 2^64
B=0x0000000000000000000000010000000000000000
BB="0x0000000000000000000000010000000000000000"
C=0xFFFFFFFFFFFFFFFF
CC="0xFFFFFFFFFFFFFFFF"
D=0x10000000000000000
DD="0x10000000000000000"
# 2^64-1
E=18446744073709551615
EE="18446744073709551615"
# 2^64
F=18446744073709551616
FF="18446744073709551616"

Expected output

apiVersion: v1
data:
  A: "0x000000000000000000000000FFFFFFFFFFFFFFFF"
  AA: '"0x000000000000000000000000FFFFFFFFFFFFFFFF"'
  B: "0x0000000000000000000000010000000000000000"
  BB: '"0x0000000000000000000000010000000000000000"'
  C: "0xFFFFFFFFFFFFFFFF"
  CC: '"0xFFFFFFFFFFFFFFFF"'
  D: "0x10000000000000000"
  DD: '"0x10000000000000000"'
  E: "18446744073709551615"
  EE: '"18446744073709551615"'
  F: "18446744073709551616"
  FF: '"18446744073709551616"'
kind: ConfigMap
metadata:
  name: test-config-mc824hgfhb

Actual output

apiVersion: v1
data:
  A: "0x000000000000000000000000FFFFFFFFFFFFFFFF"
  AA: '"0x000000000000000000000000FFFFFFFFFFFFFFFF"'
  B: 0x0000000000000000000000010000000000000000
  BB: '"0x0000000000000000000000010000000000000000"'
  C: "0xFFFFFFFFFFFFFFFF"
  CC: '"0xFFFFFFFFFFFFFFFF"'
  D: 0x10000000000000000
  DD: '"0x10000000000000000"'
  E: "18446744073709551615"
  EE: '"18446744073709551615"'
  F: "18446744073709551616"
  FF: '"18446744073709551616"'
kind: ConfigMap
metadata:
  name: test-config-mc824hgfhb

(notice how B and D (same number, 2^64, with/without leading zeros) are unquoted, but F (same number 2^64 in decimal) or C (2^64-1, fits in 64 bits) do have quotes)

Kustomize version

5.2.1

Operating system

MacOS

rubenhak commented 1 year ago

@renaudguerin, It also feels like there is an issue in kubectl/apiserver too. I think it is worth cross posting here too.

Here is the way how OpenAPI Specs for data field are defined in ConfigMap:

"data": {
        "additionalProperties": { "default": "", "type": "string" },
        "type": "object"
}

Value type should be a string (not number or boolean). For example this config fails to apply. And it should, but the ones you mentioned pass.

apiVersion: v1
data:
  foo: 1234
kind: ConfigMap
metadata:
  name: test-config

natasha41575 commented 12 months ago

/assign @stormqueen1990 for reproduction and triage

stormqueen1990 commented 11 months ago

I am able to reproduce this issue. I took a look and it appears to be related to the logic in the YAML encoder that doesn't tag this specific value as requiring quotes around it: https://github.com/kubernetes-sigs/yaml/blob/c3772b51db126345efe2dfe4ff8dac83b8141684/goyaml.v3/encode.go#L441-L469

This probably explains why it also happens with kubectl.

I haven't had the time to look if it can be patched somehow, planning to continue looking at it in a few days.

/triage accepted

DanielHilton commented 9 months ago

It is worth adding that a use case for this error is the configuration of cryptocurrency wallet addresses, which commonly begin with 0x and are misinterpreted in this way.

kubernetes-sigs / kustomize