Conversion between JSON & YAML is broken in some cases

[antonosmond]

Hi

I've found strange bug where in some scenarios, valid JSON isn't converted back to valid YAML. My use case was to replace a yaml block in one file with a yaml block from another file. My command was like this:

yq -y ".spec.jobTemplate.spec.template = $(yq '.spec.template' file_b.yaml)" file_a.yaml

Without the -y flag I get valid JSON but if I add the -y flag to output yaml, a new line is being inserted where it shouldn't be causing broken YAML.

Here is a snippet of the outputted JSON:

"args": [
  "/bin/bash",
  "-c",
  "cp -v -r -L /etc/gitlab/.s3cfg $HOME/.s3cfg && while sleep 3600; do :; done"
],

And when I add the -y flag I get this YAML:

args:
- /bin/bash
- -c
- cp -v -r -L /etc/gitlab/.s3cfg $HOME/.s3cfg && while sleep 3600; do
  :; done

It's added a new line inside that last array element.

[antonosmond]

I'll add a concrete example and details when I get a chance

[antonosmond]

Here's an example. I've removed the majority of the yaml from the files so it's easy to reproduce. Given the following yaml files:

# one.yaml
spec:
  jobTemplate:
    spec:

# two.yaml
spec:
  template:
    spec:
      containers:
        - name: task-runner
          args:
            - cp -v -r -L /etc/gitlab/.s3cfg $HOME/.s3cfg && while sleep 3600; do :; done # alpine sleep has no infinity

I want to set spec.jobTemplate.spec in one.yaml equal to spec.template from two.yaml. Using the following command should achieve that:

yq -y ".spec.jobTemplate.spec = $(yq '.spec.template' two.yaml)" one.yaml

But it causes the line to get broken. If you drop the -y flag and inspect the JSON it's fine and valid. Interestingly, if you set the value one level shallower i.e.:

yq -y ".spec.jobTemplate = $(yq '.spec.template' two.yaml)" one.yaml

instead of

yq -y ".spec.jobTemplate.spec = $(yq '.spec.template' two.yaml)" one.yaml

it works without breaking the line.

[kislyuk]

There is nothing "broken" about the YAML emitted in your test case. As described in the spec, YAML folds whitespace in strings; in other words, unless you escape your whitespace with \, it will be compressed into a single space, and lines will be wrapped.

As described in the yq help, you can get the behavior that you want by setting the -w/--width option to a large value, for example:

yq -w 99 -y ".spec.jobTemplate.spec = $(yq '.spec.template' two.yaml)" one.yaml