jqlang / jq

Command-line JSON processor
https://jqlang.github.io/jq/
Other
29.59k stars 1.54k forks source link

Is it possible to output multiple values on a single line? #785

Closed Reino17 closed 9 years ago

Reino17 commented 9 years ago

jq '.object1,.object2,.object3' returns the value for these objects each on a new line:

"value1"
"value2"
"value3"

Is there an option, or some filter to output these values on a single line, seperated by commas (or another character): "value1","value2","value3"? Or in other words: is it possible to slurp the output?

pkoppstein commented 9 years ago

Consider first:

$ jq -n -r '[1,2,3] | @csv'
1,2,3

That is probably very close to what you want, but if you want more control, consider:

$ jq -n -r '[1,2,3] | reduce .[1:][] as $i ("\(.[0])"; . + ", \($i)" )'
1, 2, 3

By the way, some people recommend asking jq usage questions at https://stackoverflow.com/questions/tagged/jq

Reino17 commented 9 years ago
jq -r "[.object1,.object2,.object3] | @csv" sample.json
"value1","value2","value3"

I'd like some more control, because I don't want the double quotes, but the output is already raw. But I'm on Windows, and I get the following error straight from the command-line:

jq -r "[.object1,.object2,.object3] | reduce .[1:][] as $i ("\(.[0])"; . + ", \($i)" )" sample.json
error: syntax error, unexpected INVALID_CHARACTER
[.object1,.object2,.object3] | reduce .[1:][] as $i (\(.[0]); . + ,
              1 compile error

I guess there's something wrong with the syntax of \($i)" )" When I put the command above in for-loop in batchscript: "; . + " was unexpected at this time.

Perhaps I should ask on StackOverflow.

pkoppstein commented 9 years ago

If you have access to a recent version of jq with "@tsv", then you could consider using that:

$ jq -n -r '["abc","def"] | @tsv' 
abc def

Otherwise, you can use string interpolation along the following lines:

$ jq -n -r '["abc","def"] | "\(.[0]) \(.[1])"'
abc def

The quoting in the above would of course have to be adapted for Windows. You might find it simpler to use jq -n -f FILENAME.

Reino17 commented 9 years ago

I'm still a novice programmer, but I think I finally figured it out. For the command-shell double quotes need to be escaped with a backslash, but once in a for-loop in a batch-script closing parenthesis also need to be escaped (with the usual circumflex) to prevent the for-loop from closing. Thus:

jq-syntax:    jq -r '[.object1,.object2,.object3] | "\(.[0]) \(.[1]) \(.[2])"'
cmd-shell:    jq -r "[.object1,.object2,.object3] | \"\(.[0]) \(.[1]) \(.[2])\""
batch-script: jq -r "[.object1,.object2,.object3] | \"\(.[0]^) \(.[1]^) \(.[2]^)\""

jq-syntax:    jq -r '[.object1,.object2,.object3] | reduce .[1:][] as $i ("\(.[0])"; . + ",\($i)")'
cmd-shell:    jq -r "[.object1,.object2,.object3] | reduce .[1:][] as $i (\"\(.[0])\"; . + \",\($i)\")"
batch-script: jq -r "[.object1,.object2,.object3] | reduce .[1:][] as $i (\"\(.[0]^)\"; . + \",\($i^)\")"

Seems logical, but caused me lots of headaches! Oh btw, for some reason the comma in the "reduce" command, to separate each value, is ignored in the batch-script. The values are separated by a single space. Not a show-stopper, but still weird though.

If you have access to a recent version of jq with "@tsv",...

By "recent" I've got a hunch you don't mean the jq 1.4 executable, because it returns jq: error: tsv is not a valid format. If @tsv does the same as the 2 above, could you tell me where to get such a recent version?

pkoppstein commented 9 years ago

@corone17 wrote:

I think I finally figured it out.

Good, though I still think that getting jq to write your "SET X=Y" commands and then using CALL might be the way to go.

could you tell me where to get such a recent version?

The git repository for jq is https://github.com/stedolan/jq but since you say you're a novice programmer, you might find it very difficult to build jq on Windows from source.

UPDATE: Thanks to @nicowilliams, https://github.com/stedolan/jq/releases includes a Windows version of jq-1.5rc1, which does include @tsv.

The choco maintainer of jq is https://chocolatey.org/profiles/svnpenn -- maybe you could ask him to make a more recent version available. The "master" version of jq has many great things that are not in version 1.4 (e.g. support for regular expressions).

nicowilliams commented 9 years ago

Try these: https://github.com/stedolan/jq/releases

Reino17 commented 9 years ago

@pkoppstein wrote:

Good, though I still think that getting jq to write your "SET X=Y" commands and then using CALL might be the way to go.

I'll keep experimenting. Thanks a lot for your help!

@nicowilliams wrote:

Try these: https://github.com/stedolan/jq/releases

Would it be possible to compile a win32 version?

nicowilliams commented 9 years ago

Ah, I should do that, yes.

Reino17 commented 9 years ago

I'm sorry, but when do you have time to compile one? Or do I have to wait untill v1.5?

nicowilliams commented 9 years ago

@corone17 I've uploaded a 1.5rc1 win32 executable. Try it.

Reino17 commented 9 years ago

Succes! jq -r "[.object1,.object2,.object3] | @tsv" works now and is much easier than the other two. The only difference I noticed, besides the tab separation of course, is that it doesn't return "null" when an object isn't found. No problem though. Thanks a lot for your help, @nicowilliams and @pkoppstein!

cf3b7S commented 8 years ago

I think the tsv way is not pretty enough. I would rather use stdin|jq '.key1, .key2' -r|xargs -n2 echo|awk ... to process my data. When using comma separator, I put one line of json string, but it output multi lines. It is more acceptable when output is inline.

xenoterracide commented 6 years ago
[root@ip-192-169-0-177 ~]# aws autoscaling describe-auto-scaling-groups |jq -r '.AutoScalingGroups[]| select( .Tags[].Value == "playground").Instances[].InstanceId '
i-080eda9cecb989c10
i-0827a09caaa7b0e89
i-0875550755d05b948
[root@ip-192-169-0-177 ~]# aws autoscaling describe-auto-scaling-groups |jq -r '.AutoScalingGroups[]| select( .Tags[].Value == "playground").Instances[].InstanceId |@tsv'
jq: error (at <stdin>:120): string ("i-080eda9c...) cannot be tsv-formatted, only array
[root@ip-192-169-0-177 ~]# aws autoscaling describe-auto-scaling-groups |jq -r '.AutoScalingGroups[]| select( .Tags[].Value == "playground").Instances[].InstanceId | @tsv'
jq: error (at <stdin>:120): string ("i-080eda9c...) cannot be tsv-formatted, only array
[root@ip-192-169-0-177 ~]# jq --version
jq-1.5

what I think I really want though is jq --join " " (make join take a stringy argument, in this case I give it a space.

this however works

aws autoscaling describe-auto-scaling-groups |jq -r '.AutoScalingGroups[]| select( .Tags[].Value == "playground").Instances[].InstanceId' |paste -s -d" "
tartley commented 5 years ago

There's an alt technique I'm using whereby I grab three fields using jq, but then use bash afterwards to put them on a single line:

myprocess | jq -r '.id, .crn, .date' | (
    while read id; do
        read crn
        read date
        echo $id $crn $date
    done
)

I haven't tested it much but it looks like it's working for me

Reino17 commented 5 years ago

The following wouldn't make me very popular here, I guess, but right after I closed this issue someone introduced me to xidel and I haven't touched jq ever since. Xidel is a HTML/XML/JSON parser (using CSS, XPath, XQuery, JSONiq and pattern templates). With Xidel it's as simple as this to concat json values (or strings):

$ echo '{"a":1,"b":2,"c":3}' | xidel -s - -e 'join($json/(a,b,c))'
1 2 3

Or comma separated:

$ echo '{"a":1,"b":2,"c":3}' | xidel -s - -e 'join($json/(a,b,c),",")'
1,2,3
bb010g commented 4 years ago

@Reino17 The equivalent jq for that Xidel off the top of my head is:

$ printf '%s\n' '{"a":1,"b":2,"c":3}' | jq -r '[.a,.b,.c]|join(" ")'
1 2 3
$ printf '%s\n' '{"a":1,"b":2,"c":3}' | jq -r '[.a,.b,.c]|join(",")'
1,2,3

If you want JSON output for items like strings here, then tossing in a map(@json) works:

$ printf '%s\n' '{"a":"red","b":"green","c":"blue"}' | jq -r '[.a,.b,.c]|join(" ")'
red green blue
$ printf '%s\n' '{"a":"red","b":"green","c":"blue"}' | jq -r '[.a,.b,.c]|map(@json)|join(" ")'
"red" "green" "blue"
dragon788 commented 4 years ago

Thanks @pkoppstein for your example, I needed functionality equivalent to @tsv but I was stuck with jq-1.4, but I was able to make a simple change that worked beautifully.

echo '{ "some": "thing", "json": "like" }' | jq -r 'to_entries[] | "\(.key)\t\(.value)"'

# Output is tab separated
some    thing
json    like