Would toml be supported?

[joshcangit]

Please describe your feature request.

I wish I could use yq to work with toml files.

Note:

• how to questions should be posted in the discussion board and not raised as an issue.
• V3 will no longer have any enhancements.

Describe the solution you'd like

If we have example.toml like:

country = "Australia"

And we run a command:

yq -i toml '.country' example.toml

it could output

"Australia"

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

I've tried another CLI to do this. https://github.com/rossmacarthur/aq But that one could only get input from stdin.

cat example.toml | aq -i toml '.country'

Additional context

There's another project of the same name. https://github.com/kislyuk/yq		mikefarah/yq	kislyuk/yq
Language	Go	Python
Dependencies	From go.mod	jq and from setup.py
json	yq . input.json	yq . input.json
yaml	yq . input.yml	yq -Y . input.yml
xml	yq -p=xml -o=xml . input.xml	xq -x . input.xml
toml		tomlq -t . input.toml
csv	yq -o=csv input.csv
tsv	yq -o=tsv input.tsv
properties	yq -p=props -o=props input.properties

Neither seem able to query csv, tsv or properties.

[mjperrone]

+1, I thought I had tomlq installed because I read the docs there (https://github.com/kislyuk/yq#toml-support) but I have this one so I am lacking the functionality. I'd love if this were supported here as well

[mikefarah]

Okay cool - I'll look into a toml go library 👍🏼

[mikefarah]

Hmm the problem I'm having with the available toml libraries in go is that they are lossy. They will not only drop comments, but also the order of the keys in maps :(

[mofosyne]

Adding support to this ticket, another use case I've been hoping for a toml editor is for easier editing of /etc/samba/smb.conf in my automated raspberry pi print server configuration script I was writing.

https://gist.github.com/mofosyne/6baab7509ccd93f74d3fa225ea57d75d

In the absence of a suitable tool, I ended up using the diff and patch command, but that will likely break if there is a change to the smb.conf file in the future (but shall do for now). This feature would allow such script to be a little bit more robust.

[zamentur]

@mofosyne alternatively for this use case you can reuse ynh_write_var_in_file bash function from yunohost (AGPLv3): https://github.com/YunoHost/yunohost/blob/dev/helpers/utils#L523 https://github.com/YunoHost/yunohost/blob/dev/helpers/utils#L600

it allows to do a lot of simple edition in a lot of different file format.

[bitdivine]

@mikefarah It sounds as if the blocker is still the lossy go library. Is it worth someone making a non-lossy library or modifying an existing library to make it non-lossy? Did you find a library that you found particularly promising that you would prefer to use, if only it were not lossy?

If we use an existing library and improve it, we could start with lossy functionality and gradually improve until toml it is on a par with yaml in terms of functionality.

[bitdivine]

That said, the premier python comment-preserving toml parser is 1000 lines. Might not be that hard to convert straight to go. We could keep that option open as well. I haven't written go in a while and am rather overloaded at the moment but I could have a shot when I have time (children and startups permitting and currently on 2 hours of sleep but one can dream of a time with time). This would help me so would also save time. https://github.com/uiri/toml/blob/3f637dba5f68db63d4b30967fedda51c82459471/toml/decoder.py

[mikefarah]

I've made some progress with @pelletier toml library here: https://github.com/mikefarah/yq/pull/1439 - I haven't looked at it since last year. From memory I think the parsing is now working reasonably well - but it will need some work to output back and remember which bits are array tables and so forth. @pelletier and I are both keen to get it working - if someone wanted to help that'd be appreciated...

[bitdivine]

I installed go and reminded myself about how it works, then cloned your branch, had a read through CONTRIBUTING.md and played with your code. I don't understand everything in the PR yet but I do note that even as it is, it is already useful. E.g. one of my original use cases was extracting some keys from a toml file to warm corresponding cache entries. The binary built in that branch already works perfectly for that use case.

[bitdivine]

Do you have specific examples of things that currently don't work, that are needed for, say, initial read-only support?

[mikefarah]

I'm planning on releasing a readonly version next release :)

[mikefarah]

4.33.1 has support for reading TOML!

[tspearconquest]

Wow, I was just looking earlier this week for a toml cli that worked like yq. Thank you, this is going to be very helpful!

[mcandre]

Good progress so far :)

toml is not yet supported as an output format

Next, would love to see this added as a built-in capability as well.