jackson-jq
Pure Java jq Implementation for Jackson JSON Processor
Usage
First, you need Java 8 or later.
If you use Maven, add the following snippet to the <dependencies> section of your POM. For instructions for other build tools (Gradle, etc.), visit jackson-jq on search.maven.org.
<dependency>
<groupId>net.thisptr</groupId>
<artifactId>jackson-jq</artifactId>
<version>1.0.0</version>
</dependency>See jackson-jq/src/test/java/examples/Usage.java for the API usage.
Using a jackson-jq command line tool
To test a query quickly, we provide jackson-jq CLI.
Please note that jackson-jq is a Java library and the CLI is provided solely for debugging/testing purpose (and not for production). The command-line options might change without notice.
$ curl -LO https://repo1.maven.org/maven2/net/thisptr/jackson-jq-cli/1.0.0/jackson-jq-cli-1.0.0.jar
$ java -jar jackson-jq-cli-1.0.0.jar --help
usage: jackson-jq [OPTIONS...] QUERY
-c,--compact compact instead of pretty-printed output
-h,--help print this message
--jq <arg> specify jq version
-n,--null-input use `null` as the single input value
-r,--raw output raw strings, not JSON texts
$ java -jar jackson-jq-cli-1.0.0.jar '.foo'
{"foo": 42}
42To test a query with a specific jq version,
$ java -jar jackson-jq-cli-1.0.0.jar --jq 1.5 'join("-")'
["1", 2]
jq: error: string ("-") and number (2) cannot be added
$ java -jar jackson-jq-cli-1.0.0.jar --jq 1.6 'join("-")' # jq-1.6 can join any values, not only strings
["1", 2]
"1-2"Homebrew (or Linuxbrew) users can alternatively run brew tap eiiches/jackson-jq && brew install jackson-jq to install the CLI. jackson-jq will be available on your $PATH.
Branches and versioning
There are currently two development branches.
develop/1.x: This branch (you are viewing), which is currently under development for the future 1.0 release. You can find preview releases at Releases page (tags:1.0.0-preview.yyyyMMdd). Although the API is not stable yet, I recommend new users to use these releases insetad of 0.x versions, because these releases have more features, better compatibility, and better performance.develop/0.x: The development branch for 0.x versions. Features that need breaking API changes will no longer be added. Go to Releases and find the latest 0.x.y version.
PRs can be sent to any of the develop/* branches. The patch will be ported to the other branch(es) if necessary.
We use Semantic Versioning 2.0.0 for Java API versioning, 1.0.0 onwards. A jq behavior fix (even if it may possibly affect users) will not be considered a major change if the fix is to make the bahavior compatible with ./jq; these kind of incompatible changes are documented in the release note.
If you get different results between ./jq and jackson-jq, please file an issue. That is a bug on jackson-jq side.
Implementation Status
jackson-jq aims to be a compatible jq implementation. However, not every feature is available; some are intentionally omitted because thay are not relevant as a Java library; some may be incomplete, have bugs or are yet to be implemented.
List of Features
Click to see the list
This table illustrates which features (picked from jq-1.5 manual) are supported and which are not in jackson-jq. We try to keep this list accurate and up to date. If you find something is missing or wrong, please file an issue. | Language Features / Functions | jackson-jq | |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------| | [Basic filters](https://stedolan.github.io/jq/manual/v1.5/#Basicfilters) | ○ | | • [`.`](https://stedolan.github.io/jq/manual/v1.5/#.) | ○ | | • [`.foo`, `.foo.bar`](https://stedolan.github.io/jq/manual/v1.5/#.foo,.foo.bar) | ○ | | • [`.foo?`](https://stedolan.github.io/jq/manual/v1.5/#.foo?) | ○ | | • [`.[
Known Compatibility Issues / Differences
Category: BUG
(*11) Operator Precedences in 1 + 3 as $a | ($a * 2)
##### Description
The presence of `as $a` affects precedence of `|` and other operators in jq:
```console
$ jq -n '1 + 3 | (. * 2)' # interpreted as (1 + 3) | (. * 2)
8
$ jq -n '1 + 3 as $a | ($a * 2)' # interpreted as 1 + (3 as $a | ($a * 2))
7
```
whereas jackson-jq consistently interprets them as `(1 + 3)` whether `as $a` is used or not:
```console
$ java -jar jackson-jq-cli-1.0.0.jar -n '1 + 3 | (. * 2)' # interpreted as (1 + 3) | (. * 2)
8
$ java -jar jackson-jq-cli-1.0.0.jar -n '1 + 3 as $a | ($a * 2)' # interpreted as (1 + 3) as $a | ($a * 2)
8
```
##### Examples
```console
$ jq -n '1 + 3 as $a | ($a * 2)' # interpreted as 1 + (3 as $a | ($a * 2))
7
$ java -jar jackson-jq-cli-1.0.0.jar -n '1 + 3 as $a | ($a * 2)' # interpreted as (1 + 3) as $a | ($a * 2)
8
```
##### Workaround
Use explicit parentheses.
##### Links
* [jackson-jq#72](https://github.com/eiiches/jackson-jq/issues/72)
(*3) Multiple functions with the same name in the same scope
##### Description If the function with the same is defined more than once at the same scope, jackson-jq uses the last one. ##### Examples ```console $ jq -n 'def f: 1; def g: f; def f: 2; g' 1 $ java -jar jackson-jq-cli-1.0.0.jar -n 'def f: 1; def g: f; def f: 2; g' 2 ``` ##### Workaround Avoid using the duplicate function name. ```console $ java -jar jackson-jq-cli-1.0.0.jar -n 'def f1: 1; def g: f1; def f2: 2; g' 1 ```Category: BY DESIGN
(*1) Error Message Wording
##### Description Error messages differ between jq and jackson-jq and they also tend to change between versions. ##### Workaround None. This is by design and will not be fixed.(*6) env/0 is not available by default.
##### Description
`env/0` is not available by default for security reasons and must be added manually to the scope.
##### Workaround
Add `env/0` manually into the scope:
```java
SCOPE.addFunction("env", 0, new EnvFunction())
```
(*4) Field Ordering in JSON Object
##### Description The order of the keys in JSON is not preserved. It was a design decision but we are slowly trying to fix this in order to improve the compatibility with jq. ##### Workaround None. Use array if the order is important.(*5) 0 / 0 is an error in jackson-jq.
##### Description
jq evaluates `0 / 0`, if hard-coded, to NaN without any errors, whereas `0 | 0 / .` results in a zero-division error. jackson-jq always raises an error in both cases.
##### Examples
```console
$ jq -n '0 / 0'
null
$ jq -n '10 / 0'
jq: error: Division by zero? at (*8) ... |= empty is an error in jackson-jq.
##### Description
`.foo |= empty` always throws an error in jackson-jq instead of producing an unexpected result. jq-1.5 and jq-1.6 respectively produces a different and incorrect result for `[1,2,3] | ((.[] | select(. > 1)) |= empty)`. [jq#897](https://github.com/stedolan/jq/issues/897) says "empty in the RHS is undefined". You can still use `_modify/2` directly if you really want to emulate the exact jq-1.5 or jq-1.6 behavior.
##### Examples
```console
$ jq-1.6 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
[
1,
3
]
$ jq-1.5 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
null
$ jq-1.2 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
[
1,
2,
3
]
$ java -jar jackson-jq-cli-1.0.0.jar --jq 1.6 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
jq: error: `|= empty` is undefined. See https://github.com/stedolan/jq/issues/897
$ java -jar jackson-jq-cli-1.0.0.jar --jq 1.5 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
jq: error: `|= empty` is undefined. See https://github.com/stedolan/jq/issues/897
```
##### Workaround
You can use `_modify/2` if you really want to the original behavior.
```console
$ java -jar jackson-jq-cli-1.0.0.jar --jq 1.6 -n '[1,2,3] | _modify((.[] | select(. > 1)); empty)'
[ 1, 3 ]
$ java -jar jackson-jq-cli-1.0.0.jar --jq 1.5 -n '[1,2,3] | _modify((.[] | select(. > 1)); empty)'
null
```
(*7) Variables don't carry path information even in jq 1.5 compat mode.
##### Description `path(.foo as $a | $a)` always throws an error as $variables in jackson-jq do not carry path information like jq-1.5 accidentally? did. The behavior is fixed in jq-1.6 whose [documentation](https://stedolan.github.io/jq/manual/v1.6/#Assignment) explicitly states them as "not a valid or useful path expression". So, I dicided not to implement it even in jq-1.5 compatible mode. ##### Examples jq 1.5 ```console $ jq-1.5 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}' ["foo"] $ java -jar jackson-jq-cli-1.0.0.jar --jq 1.5 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}' jq: error: Invalid path expression with result 1 ``` jq 1.6 ```console $ jq-1.6 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}' jq: error (at(*2) try (break $label) catch . always produces {"__jq": 0}.
##### Description
try (break $label) catch . always produces {"__jq": 0} in jackson-jq, while `__jq` should contain the index of the label the `break` statement jumps to.
##### Examples
```console
$ jq -n 'label $a | label $b | try (break $b) catch .'
{
"__jq": 1
}
$ java -jar jackson-jq-cli-1.0.0.jar -n 'label $a | label $b | try (break $b) catch .'
{
"__jq" : 0
}
```
##### Workaround
None. Tell us your use case if you need this feature.
Category: BUGFIX
(*9) indices("") returns [] (empty array) in jackson-jq.
##### Description
`indices/1` implementation in jq-1.5 and jq-1.6 had a bug that caused `indices("")` to end up in infinite loop which eventually leads to OOM. The bug is [fixed](https://github.com/stedolan/jq/commit/2660b04a731568c54eb4b91fe811d81cbbf3470b) and likely to be in jq-1.7 (not released yet). jackson-jq chose not to simulate this bug.
##### Examples
```console
$ jq-1.5 -n '"x" | indices("")' # stuck in infinite loop
^C
$ jq-1.6 -n '"x" | indices("")' # stuck in infinite loop
^C
$ jq-1.6-83-gb52fc10 -n '"x" | indices("")'
[]
$ java -jar jackson-jq-cli-1.0.0.jar -n '"x" | indices("")'
[ ]
```
Using jackson-jq/extras module
The jackson-jq/extras module is a jq module that provides some useful functions that do not exist in jq.
To use this module, you need to add the following Maven dependency and set BuiltinModuleLoader (see jackson-jq/src/test/java/examples/Usage.java) to the scope.
<dependency>
<groupId>net.thisptr</groupId>
<artifactId>jackson-jq-extra</artifactId>
<version>1.0.0</version>
</dependency>Now, you can import the module in jq:
import "jackson-jq/extras" as extras;
extras::uuid4For a historical reason, adding the Maven dependency also makes the functions directly available to jq. This behavior is deprecated and will be removed at some point in the future.
List of Functions
#### uuid4/0 - `jackson-jq -n 'uuid4'` #=> `"a69cf146-f40e-42e1-ae88-12590bdae947"` #### random/0 - `jackson-jq -n 'random'` #=> `0.43292159535427466` #### timestamp/0, strptime/{1, 2}, strftime/{1, 2} - `jackson-jq -n 'timestamp'` #=> `1477162056362` - `jackson-jq -n '1477162342372 | strftime("yyyy-MM-dd HH:mm:ss.SSSXXX")'` #=> `"2016-10-23 03:52:22.372+09:00"` - `jackson-jq -n '1477162342372 | strftime("yyyy-MM-dd HH:mm:ss.SSSXXX"; "UTC")'` #=> `"2016-10-22 18:52:22.372Z"` - `jackson-jq -n '"2016-10-23 03:52:22.372+09:00" | strptime("yyyy-MM-dd HH:mm:ss.SSSXXX")'` #=> `1477162342372` - `jackson-jq -n '"2016-10-22 18:52:22.372" | strptime("yyyy-MM-dd HH:mm:ss.SSS"; "UTC")'` #=> `1477162342372` #### uriparse/0 - `jackson-jq -n '"http://user@www.example.com:8080/index.html?foo=1&bar=%20#hash" | uriparse'` #=> ```json { "scheme" : "http", "user_info" : "user", "raw_user_info" : "user", "host" : "www.example.com", "port" : 8080, "authority" : "user@www.example.com:8080", "raw_authority" : "user@www.example.com:8080", "path" : "/index.html", "raw_path" : "/index.html", "query" : "foo=1&bar= ", "raw_query" : "foo=1&bar=%20", "query_obj" : { "bar" : " ", "foo" : "1" }, "fragment" : "hash", "raw_fragment" : "hash" } ``` #### uridecode/0 - `jackson-jq -n '"%66%6f%6f" | uridecode'` #=> `"foo"` #### hostname/0 - `jackson-jq -n 'hostname'` #=> `"jenkins-slave01"`Contributing
- If you are planning to send a PR and the change is not small, please open an issue and discuss it with the authors first.
- Other than bug reports or patches, documentation improvements (including small grammatical or wording corrections) would be greatly appreciated.
License
This software is licensed under Apache Software License, Version 2.0, with some exceptions:
- jackson-jq/src/test/resources contains test cases from stedolan/jq.
- jackson-jq/src/main/resources/net/thisptr/jackson/jq/jq.json contains function definitions extracted from stedolan/jq.
See COPYING for details.