Why is yq putting multiline separator indicator (`|-`) to complex object replacements?

[Skarlso]

Describe the bug OSX

Version: v4.40.5

cat test.yaml
test:
  this: cool
➜ yq -i '.test.this = "bla: bla\ntwice: bla"' test.yaml
➜ cat test.yaml
test:
  this: |-
    bla: bla
    twice: bla

Why did the |- get there and how can I remove it?

[nicktimko]

The program is pretty-printing the equivalent string with the newline. See https://yaml-multiline.info/ for how the multiline marks work, but the actual value is identical.

$ echo '{"string": "x\\ny"}' | yq -P
string: |-
  x
  y

$ echo '{"string": "x\\ny"}' | yq -P | yq -ojson
{
  "string": "x\ny"
}

If you have jq output the raw string, you see it actually includes a newline (0x0A)

$ echo '{"string": "x\\ny"}' | jq -r .string | xxd
00000000: 780a 790a                                x.y.

If you want an actual \n in the string and not a newline, it seems like a bug that yq won't accept the backslash escape (which would eat up the backslashes for the newline escape):

$ # yq ignores \\ and instead translates \n into 0x0A, so the true output has a backslash + newline
$ echo '{"string": "x"}' | yq '.string = "x\\ny"' | yq -r .string | xxd
00000000: 785c 0a79 0a                             x\.y.

$ # jq accepts \\n and puts literal '\n' in the output
$ echo '{"string": "x"}' | jq '.string = "x\\ny"' | jq -r .string | xxd
00000000: 785c 6e79 0a                             x\ny.

[nicktimko]

Maybe related: #1692 #1814

[Skarlso]

Bummer. Thanks for the info @nicktimko!

[nicktimko]

@Skarlso did you intend to put the escaped string into the YAML (it contains the characters \ and n, not a newline)?

[Skarlso]

Yes, that's an object. Something like

bla: value1
bla2: value2

Which is the replacement. That should be translated into a newline I believe?

If I would actually try and do a multi-line string I would get a weird |2 there.

yq -i '.test.this |= "    bla: bla
    twice: bla
"' test.yaml
➜  SAP cat test.yaml
test:
  this: |2
        bla: bla
        twice: bla

[Skarlso]

Ah sorry, my bad, too many spaces. This is the equivalent.

yq -i '.test.this |= "bla: bla
twice: bla
"' test.yaml
➜  SAP cat test.yaml
test:
  this: |
    bla: bla
    twice: bla

Still got | in there...

[mikefarah]

The | |- and other variations are how multi line strings are encoded in yaml. Unfortunately, the underlying yaml encoder does not give me many controls to be able to force particular variations; and sometimes it 'auto' formats things :(

It's a known issue against go-yaml; and there are a bunch of tickets against it - but no one has worked on that repo for many months now :(