Prettify mode limits, questions, and plans

[vaab]

Hi, I've been using libyaml, yq, and libfyaml, and I noticed the existence of the pretty mode in libfyaml, which might not be exactly what I expected it was, so I wonder what are the intentions and responsibility of this mode. Let me clarify also my understanding to this point. I know there are many inconsistencies between tools, some are intentional, and others are bugs or functionality yet to be implemented.

Let's start with creating this yaml code:

cat <<EOF > /tmp/test.yaml
%YAML 1.1
%TAG !e! tag:example.com,2000:app/
---
!!map {
  ? !!str "sequence"
  : !!seq [ !e!bar "one", !!str "two", 1, 3.0, !!str "2010-01-01" ],
  ? !!str "mapping"
  : !!map {
    ? !!str "sky" : !myobj "blue",
    ? !!str "sea" : !!foo "green",
  },
}
EOF

Notice several points:

• The usage of a tag directive in a value !e!bar
• The usage of local !myobj tag.
• long json version of maps and sequence using brackets {}, []
• The usage of !! is refering here to standard prefix, and to take one example !!str is the default type of a scalar that would be parsed as a string.
• Notice that the scalar whose raw content would be solely "2010-01-01" could be considered a !!timestamp (for yq for instance and not a !!str, and a !!date for libyaml). More on that later.

Here is the prettified version of this code using yq -P /tmp/test.yaml:

%YAML 1.1
sequence:
  - !<tag:example.com,2000:app/bar> one
  - two
  - 1
  - 3.0
  - "2010-01-01"
mapping:
  sky: !myobj blue
  sea: !!foo green

Here is the prettified version of this code using cat /tmp/test.yaml | shyaml get-value (this is a python wrapper around libyaml):

sequence:
- !<tag:example.com,2000:app/bar> one
- two
- 1
- 3.0
- '2010-01-01'
mapping:
  sky: !myobj blue
  sea: !!foo green

We notice that it is very similar, and that's encouraging. And they went both through:

• resolution of tag directive to get the real tag value preserved
• keeping the quotes (single or double quote) around the forced !!str type of "2010-01-01" that would be interpreted as a !!timestamp for yq, or a !!date for libyaml if the !!str and the double quotes where removed.
• keeping intact !myobj and !!foo
• removal of double quotes where they are redundants

Using fy-tool --dump -mode pretty /tmp/test.yaml:

%YAML 1.1
%TAG !e! tag:example.com,2000:app/
--- !!map
!!str "sequence": !!seq
- !e!bar "one"
- !!str "two"
- 1
- 3.0
- !!str "2010-01-01"
!!str "mapping": !!map
  !!str "sky": !myobj "blue"
  !!str "sea": !!foo "green"

We notice it is properly:

• de-jsonifying the output (removing {} around maps, and [] around sequences and using the yaml representation)

But it doesn't:

• remove !!str, !!map, !!seq where they are redundant.
• remove double quotes where they are redundant.

If keeping the tag directive doesn't sound problematic, as the YAML doc is still complete, it means that devs should not think that fy_emit_node_to_string(..) is complete by itself. They should not forget to use fy_emit_document_start(..) before.

Would you welcome some PR to move on some (or all) of these points ? Or want to share your stance on these topics ?

• pretty mode should remove !!str/!!map/... if it is redundant (beware, this means that further parsing needs to be done, as many types are not implemented yet in libfyaml AFAIK, like !!timestamp)
• pretty mode should remove double quotes if they are redundant
• a mode (in parser_cfg) should exist to ask for resolving tag directive and allow fy_emit_node_to_string(..) to be independant from a document start.

I already have some work done on these.

I'm sure also that I missed important information about these... so feel free to correct me.

[vaab]

And of course, many thanks for this new lib that is more than welcome !

[vaab]

Also, as a consequence of libfyaml not parsing for different types of scalar (AFAIK), there are no distinction made between an empty node and an empty stringed value... As food for though:

Libyaml:

$ echo "a: " | shyaml get-value -y a
null
$ echo "a: ''" | shyaml get-value -y a
''

yq:

$ echo "a: " | yq -r=false '.a'   ## humm, suprising answer

$ echo "a: " | yq '.a | type'
!!null
$ echo "" | yq '. | type'  ## but consistent
!!null
$ echo "a: ''" | yq -r=false '.a'
''
$ echo "a: ''" | yq '.a | type'
!!str

[pantoniou]

Yes, all these are a consequence of libfyaml currently only supporting the builtin schema (where all scalars are strings). There is work underway to support the default object schemas (and more) but it's not on master yet.

Leaving this open for now, and will revisit later