Closed polarathene closed 8 months ago
jq
/ yq
manipulation (eg: Aligned Columns output)With doggo
output, you can easily produce the aligned columns of your choice too (picks two fields of interest and outputs as TSV document):
# The sub operator is only to remove escaped double quote wrapping of TXT values
# Can omit, but output between tools for TSV is then inconsistent for TXT values
# jq
doggo example.test NS MX TXT A --json | jq -r '["TYPE", "VALUE"], (.[] | [.answers[0].type, .answers[0].address])
| .[1] |= sub("\""; ""; "g")
| @tsv'
# yq
doggo example.test NS MX TXT A --json \
| yq -o=tsv '[["TYPE", "VALUE"]] + [.[]
| [.answers[0].type, .answers[0].address]
| .[1] |= sub("\"", "")
]'
TYPE VALUE
NS ns1.example.test.
MX 10 mail.example.test.
TXT v=spf1 mx -all
A 172.16.42.11
I'm not sure how you'd approach that with q
, but assume it's a bit more tricky? 🤷♂️
EDIT: I figured it out for q
, but as you can see below it's much more work 😞
Process used for YAML format (JSONL is same but skips step 1):
---
, head -n -1
before sed
is not needed, step 6 filters out the empty array).hdr
section.mx
+ preference
fields with space delimiter into the mx
field and remove the preference field
.- key: value
into - [key, value]
as input required for outputting TSV.yq
command now ingests that single YAML doc and prepends the TSV headers, then outputs as TSV :tada:q example.test NS MX TXT A --format yaml \
| sed 's/^$/---/' \
| yq '. as $item ireduce ({}; [$item])
| .[].answers[] | [with_entries(select(.key != "hdr"))]
| with(select(.[] | has("mx")); .[] |= (.mx = ([.preference, .mx] | join(" "))) |= del .preference)
| with(select(.[] | has("txt")); .[].txt |= join(""))
| (.[] | to_entries | with(.[]; . |= [.key | upcase, .value]))' \
| yq -o=tsv '[["TYPE", "VALUE"]] + .'
--format json
)Here's the equivalent for the JSONL output from q
. yq
treats it as a multi-doc so no sed
needed. Other than that it's just adjusted to the PascalCase convention:
q example.test NS MX TXT A --format json \
| yq -p=json '. as $item ireduce ({}; [$item])
| .[].Answers[] | [with_entries(select(.key != "Hdr"))]
| with(select(.[] | has("Mx")); .[] |= (.Mx = ([.Preference, .Mx] | join(" "))) |= del .Preference)
| with(select(.[] | has("Txt")); .[].Txt |= join(""))
| (.[] | to_entries | with(.[]; . |= [.key | upcase, .value]))' \
| yq -o=tsv '[["TYPE", "VALUE"]] + .'
Thanks for the fabulously detailed feedback! v0.19.0 brings some changes:
All "inverted" flags (with the exception of tls-insecure-skip-verify
) have been reversed to become default true booleans. They can be toggled with the dig syntax of +noflag
or --flag=false
. I understand this is a departure from convention for certain CLI programs, but I think this strikes a balance of familiarity with DNS utilities and a more standardized unix flag format.
Pretty TTLs are now truncated when the minute or second component(s) are zero.
The structured output (JSON and YAML) format is now a list of output.Entry
s with all lowercase field names. This is consistent between JSON and YAML.
There is a new column output format:
$ q --format=column example.com
A 1h29m33s 93.184.216.34
AAAA 52m42s 2606:2800:220:1:248:1893:25c8:1946
MX 9h25m2s 0 .
NS 3h21m31s a.iana-servers.net.
NS 3h21m31s b.iana-servers.net.
TXT 9h25m2s "v=spf1 -all"
TXT 9h25m2s "wgyf8z8cgvm2qmxpnbnldrcltvk4xqfn"
Thanks for the fabulously detailed feedback!
Brilliant! Thanks for tackling the concerns raised ❤️
I'm not able to give it a spin right now but I look forward to it! 😁
Closing for lack of activity, feel free to reopen if needed.
q
is a great DNS utility although the UX has a little bit of friction. I'm not sure how much of these you can address easily while keeping the same size advantage over alternatives likedoggo
.Column Alignment
Readability could be improved if the record values were column aligned? The TTL and record types otherwise make that less pleasant to scan through.
You can kind of workaround it by requesting a specific record type, or using
--short
, but for multiple records where alignment is useful, it can be helpful to have the context of the record type.Here's how it looks from
doggo
:Column Headers
Not too important, and definitely should be optional (opt-in is fine), this seems to compliment the column alignment feature as shown in the
doggo
example above.Likewise, I don't think you can filter columns (eg: If I'm interested in the equivalent of
--short
but additionally with the record type).doggo
doesn't appear to support disabling the headers line, or columns displayed. Just a feature I've seen in some other CLI tools when presenting results.Default opt-out flags (
+[no]
)I wanted to try disable the default
--pretty-ttls
feature early on, and saw the help mention of dig notation, but that wasn't immediately obvious to me to figure out and I failed a few times until landing on this issue comment to realize the option replaces--
with+
or+no
.--pretty-ttls false
or--no-pretty-ttls
would seem a bit more natural? This particular setting could be a preference as a default, but there is no config or generic ENV support, I suppose you could use a shell alias 🤷♂️Those are nice to haves, but I think just having an example on your README with the
+no
syntax would be sufficient,--pretty-ttls
is probably a good example.Pretty TTL
In my case for 60s, the pretty TTL renders as
1m0s
, and the README example 1 day is24h0m0s
. The 0 units could maybe be truncated as they're added noise?--format json
This differs from
doggo
which actually outputs a JSON document array of records.Your JSON output is technically JSONL (JSON Lines), which can be useful too. Just thought I'd point out that difference from what I was expecting to see.
Tools like
yq
could convert the YAML format into JSON easily enough. Whilejq
can convert JSONL to a pretty printed JSON array withq example.test NS MX TXT A --format json | jq --slurp '.'
:Q JSONL output
```json {"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":15,"Class":1,"Ttl":60,"Rdlength":9},"Preference":10,"Mx":"mail.example.test."}],"ID":34374,"Truncated":false} {"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":16,"Class":1,"Ttl":60,"Rdlength":15},"Txt":["v=spf1 mx -all"]}],"ID":3010,"Truncated":false} {"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":1,"Class":1,"Ttl":60,"Rdlength":4},"A":"172.16.42.11"}],"ID":57132,"Truncated":false} {"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":2,"Class":1,"Ttl":60,"Rdlength":6},"Ns":"ns1.example.test."}],"ID":22482,"Truncated":false} ```Q JSONL => JSON output (via jq)
``` [ { "Server": "127.0.0.11:53", "QueryTime": 2000000, "Answers": [ { "Hdr": { "Name": "example.test.", "Rrtype": 15, "Class": 1, "Ttl": 60, "Rdlength": 9 }, "Preference": 10, "Mx": "mail.example.test." } ], "ID": 34374, "Truncated": false }, { "Server": "127.0.0.11:53", "QueryTime": 2000000, "Answers": [ { "Hdr": { "Name": "example.test.", "Rrtype": 16, "Class": 1, "Ttl": 60, "Rdlength": 15 }, "Txt": [ "v=spf1 mx -all" ] } ], "ID": 3010, "Truncated": false }, { "Server": "127.0.0.11:53", "QueryTime": 2000000, "Answers": [ { "Hdr": { "Name": "example.test.", "Rrtype": 1, "Class": 1, "Ttl": 60, "Rdlength": 4 }, "A": "172.16.42.11" } ], "ID": 57132, "Truncated": false }, { "Server": "127.0.0.11:53", "QueryTime": 2000000, "Answers": [ { "Hdr": { "Name": "example.test.", "Rrtype": 2, "Class": 1, "Ttl": 60, "Rdlength": 6 }, "Ns": "ns1.example.test." } ], "ID": 22482, "Truncated": false } ] ```doggo
output is a bit nicer with all lowercase field names and more predictable layout (_answers.address
provides the record value, andanswers.type
the record type, I don't need to have a mapping ofAnswers.Hdr.Rrtype
value toAnswers[$RECORD_TYPE]
_), so it's a bit easier to process:Doggo JSON output
```json [ { "answers": [ { "name": "example.test.", "type": "NS", "class": "IN", "ttl": "60s", "address": "ns1.example.test.", "status": "", "rtt": "1ms", "nameserver": "127.0.0.11:53" } ], "authorities": null, "questions": [ { "name": "example.test.", "type": "NS", "class": "IN" } ] }, { "answers": [ { "name": "example.test.", "type": "MX", "class": "IN", "ttl": "60s", "address": "10 mail.example.test.", "status": "", "rtt": "0ms", "nameserver": "127.0.0.11:53" } ], "authorities": null, "questions": [ { "name": "example.test.", "type": "MX", "class": "IN" } ] }, { "answers": [ { "name": "example.test.", "type": "TXT", "class": "IN", "ttl": "60s", "address": "\"v=spf1 mx -all\"", "status": "", "rtt": "0ms", "nameserver": "127.0.0.11:53" } ], "authorities": null, "questions": [ { "name": "example.test.", "type": "TXT", "class": "IN" } ] }, { "answers": [ { "name": "example.test.", "type": "A", "class": "IN", "ttl": "60s", "address": "172.16.42.11", "status": "", "rtt": "0ms", "nameserver": "127.0.0.11:53" } ], "authorities": null, "questions": [ { "name": "example.test.", "type": "A", "class": "IN" } ] } ] ```That's not an issue specific to JSON format though, YAML is affected by that too and I guess changing that would be a breaking change for anyone relying on the current output 😅
q
does at least have some improvement with howMX
record splits out thepreference
field. Fordoggo
splitting a string by the space char as a delimiter isn't difficult if separating the preference was required.JSON vs YAML format inconsistency with field names
JSONL aside, the JSON output is using PascalCase convention for fields, while YAML output is using flatcase.
Additionally, the YAML output has each result split by a blank line, which isn't valid YAML?
---
can be used to specify multiple YAML docs.