search

kislyuk / yq

Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
https://kislyuk.github.io/yq/
Apache License 2.0
2.58k stars 83 forks source link

Enable configurable indentation #52

Closed AnthonyWC closed 4 years ago

AnthonyWC commented 5 years ago

e.g.

foo.json

{
  "test": {
    "s1": "bar",
    "s2": [
      "snowflake",
      "s3",
      "gcloud"
    ]
  }
}

yq -y . foo.json

test:
  s1: bar
  s2:
  - snowflake
  - s3
  - gcloud

When valid yaml should be:

 test:
   s1: bar
   s2:
     - snowflake
     - s3
     - gcloud
kislyuk commented 5 years ago

That format is valid YAML.

AnthonyWC commented 5 years ago

Interesting; didn't realize the spec allowed for 0 spacing. However it seems it would fail some yaml linter; is it possible to specify a default format spacing?

kislyuk commented 5 years ago

yq currently uses PyYAML for YAML encoding. If there is an option that can be passed to PyYAML to specify the spacing, then yes, we can make that happen. Can you take a look at the PyYAML docs and give an example of the option if it exists?

AnthonyWC commented 5 years ago

From here there is some suggestions https://stackoverflow.com/questions/25108581/python-yaml-dump-bad-indentation but doesn't look like a native PyYAML option exists. Also there is a bug with indentation with lists in PyYAML (https://github.com/yaml/pyyaml/issues/234)

kislyuk commented 5 years ago

OK thanks. It looks like ruamel.yaml has that option. I'm going to take another look at switching away from PyYAML (it has a number of flaws).

gecube commented 5 years ago

Can we switch to https://github.com/crdoconnor/strictyaml ? Will it fix the issue or not? I didn't make the comparison to PyYAML yet, but it looks very promising.

kislyuk commented 5 years ago

No. It introduces too many incompatibilities with YAML features that are in common use.

kislyuk commented 5 years ago

At this point I find there is no suitable replacement for PyYAML. Therefore this bug is blocked by https://github.com/yaml/pyyaml/issues/234.

jayvdb commented 5 years ago

Previous issue regarding indentation: https://github.com/kislyuk/yq/issues/29

Previous issue regarding ruamel.yaml https://github.com/kislyuk/yq/issues/15 , which I have left at comment at as I think ruamel.yaml is stable now (it wasnt when the issue was closed) and is highly desirable.

daks commented 5 years ago

Hello, I'm facing the same problem, my YAML lists are converted from

foo:
  - bar
  - baz

to 

foo:
- bar
- baz

which introduces unecessary and hard-to-review changes. I would also like the possibility of defining custom indentation.

PS: Thanks for this useful tool

jayvdb commented 5 years ago

I did some further analysis of available tools at https://github.com/adrienverge/yamllint/issues/62#issuecomment-510811873 . In short, ruamel.yaml.cmd is a very useful tool to post-process yaml out of yq , to restore expected indentation.

It would be great if ruamel.yaml was used internally by yq for even better results. ruamel.yaml is now 1.0+ , so the API is now stable.

kislyuk commented 5 years ago

At this point I consider staying with PyYAML to be the safest course of action. PyYAML has at least 3 orders of magnitude more usage and exposure than ruamel.yaml, and recently regained tractable maintainership. This bug remains blocked by yaml/pyyaml#234.

kislyuk commented 4 years ago

I have found a way to address this issue. By default, yq -y will now indent block lists by 2 spaces (- item instead of - item). The old behavior (0 spaces, the PyYAML default) can be triggered via yq -y --indentless-lists.

Released in v2.8.0, please test.

ghost commented 4 years ago

@kislyuk it’s working for me