Provide way to escape "." in key name when using --data-value options.

Describe the problem/challenge you have

I want to use --data-value to set a key where the key name includes a ".".

Trying with various possible conventions used by other tools there currently appears to be no way of doing this:

~$ ytt -v 'data."file.yaml"=a:b' --data-values-inspect
data:
  '"file':
    yaml": a:b

~$ ytt -v 'data["file.yaml"]=a:b' --data-values-inspect
data["file:
  yaml"]: a:b

~$ ytt -v 'data['file.yaml']=a:b' --data-values-inspect
data[file:
  yaml]: a:b

~$ ytt -v "data.file\.yaml=a:b" --data-values-inspect
data:
  file\:
    yaml: a:b

This means that if a data value schema uses a "." in a key name, eg. "file.yaml", "tls.crt" or "tls.key", there is no way to set it from the command line and one has to use a data values file as input.

Note that I know you can do something like:

~$ ytt --data-value-yaml 'data={"file.yaml":"a:b"}' --data-values-inspect
data:
  file.yaml: a:b

But this only helps in case where can set the whole of data in one go. It doesn't help if a schema or default values already had data with other key/values in it, as setting like this would overwrite everything.

~$ ytt -v data.key=1 --data-value-yaml 'data={"file.yaml":"a:b"}' --data-values-inspect
data:
  file.yaml: a:b

So it has to be additive to what exists already.

Describe the solution you'd like

The yq tool provides one example of an escaping mechanism which could be used:

echo 'a:
 b.c: 2' | yq e '.a."b.c"' -

If adopted this convention then would expect:

~$ ytt -v 'data."file.yaml"=a:b' --data-values-inspect
data:
  file.yaml: a:b

In implementing this approach one would have to consider the likelihood that anyone is already using double quotes in a key name such that using the escaping mechanism as the default would change how it works for those existing users. A gut feel says it would be unlikely anyone was using quotes in key names as it would be an anti pattern for YAML in general, especially not when using ytt --data-value options via the command line.

Anything else you would like to add:

@jtigger on Slack (in the #carvel channel) pointed at the following relevant code where changes would be required.

(this next section added by @pivotaljohn)

Implementation Notes

When we pick this work up:

be consistent within Carvel: just like https://github.com/vmware-tanzu/carvel-kapp-controller/pull/744, we'll allow for escaping using the \ character.
evaluate the breaking change impact: we believe that while this is a change in behavior and therefore technically a breaking change, that the impact will be small or none. Let's verify by soliciting from the community whether anyone is deliberately expecting a trailing slash in their data values' keys (see in the description of this issue, the output from the fourth option).
when we do implement, let's be as close as possible to the kapp-controller behavior.
when we deliver, we need to explicitly call out that this is a breaking change and note the results of our "research" from step 2.

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

Hi @GrahamDumpleton 👋

Thank you for creating this issue and cross posting the link to the slack conversation. This request is definitely something that we think would add value to ytt. Also, since there is no workaround for escaping "." in key names, the priority of this request is 'important-longterm' but could be bumped up if more users support this request.

One thing that we want to consider is backwards compatibility. Enabling escape characters for data-values flags could affect users currently using flavors of --data-value flag. However, we don't have much signal of people using quotes in their ytt calls, so the risk of breaking someones process is low.

Since these commands are processed in the shell, escape characters (",',\) are not present in the data-values (as you displayed in the examples above), thus we would need to combine two escape characters to properly identify the "." in the key name as part of the key.

We propose using single quotes around the whole argument, with double quotes around key names that should not be split.

$ ytt -v 'data."file.yaml"=a:b' --data-values-inspect
data:
  file.yaml: a:b

carvel-dev / ytt

Provide way to escape "." in key name when using --data-value options. #671

Implementation Notes