mikefarah / yq

yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor
https://mikefarah.gitbook.io/yq/
MIT License
11.88k stars 585 forks source link

TOML nested array of tables with arrays breaks v4.35.1 #1758

Open mbigras opened 1 year ago

mbigras commented 1 year ago

Describe the bug

Hi! First of all, thank you so much for making and maintaining yq; I happily use your tool everyday and especially thank you for starting work on TOML support—see https://github.com/mikefarah/yq/issues/1364 issue!

Issue: I noticed that yq breaks when parsing TOML nested array of tables with arrays (lol)—see Array of Tables section. Workaround: To work around, you can parse TOML with tomljson command, then pipe JSON to yq.

Version of yq: v4.35.1,...,v4.42.1 Operating system: Mac and Linux Installed via: Homebrew and Docker

Input TOML data1.toml:

[[fruits]]
name = "apple"
[[fruits.varieties]]  # nested array of tables
name = "red delicious"

Command The command you ran:

yq -ptoml -oyaml data1.toml

Actual behavior

Error: bad file 'data1.toml': cannot index array with 'varieties' (strconv.ParseInt: parsing "varieties": invalid syntax)

Expected behavior

fruits:
  - name: apple
    varieties:
      - name: red delicious

Additional context I suspect this is a problem with yq because the tomljson command can handle data1.toml like the following shell session illustrates:

$ tomljson /dev/stdin <<'TOML' | yq -P
[[fruits]]
name = "apple"
[[fruits.varieties]]  # nested array of tables
name = "red delicious"
TOML
fruits:
  - name: apple
    varieties:
      - name: red delicious

Again, thank you for yq and please advise if I should do any other experiments.

edmorley commented 11 months ago

We're encountering this too, in: https://github.com/heroku/builder/actions/runs/6303904793/job/17114118993?pr=392#step:5:9

When running yq '.stack.run-image' builder-22/builder.toml against this toml file: https://github.com/heroku/builder/blob/b83a3606a68b882484ee3353a58c2813d321a9f4/builder-22/builder.toml

antoinedeschenes commented 5 months ago

Also seeing the issue with sub-tables, for example:

curl -s https://raw.githubusercontent.com/veertuinc/gitlab-runner/master/config.toml.example | yq -p toml -o yaml .