JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document.
Jolt :
The Stock transforms are:
shift : copy data from the input tree and put it the output tree
default : apply default values to the tree
remove : remove data from the tree
sort : sort the Map key values alphabetically ( for debugging and human readability )
cardinality : "fix" the cardinality of input data. Eg, the "urls" element is usually a List, but if there is only one, then it is a String
Each transform has its own DSL (Domain Specific Language) in order to facilitate its narrow job.
Currently, all the Stock transforms just effect the "structure" of the data. To do data manipulation, you will need to write Java code. If you write your Java "data manipulation" code to implement the Transform interface, then you can insert your code in the transform chain.
The out-of-the-box Jolt transforms should be able to do most of your structural transformation, with custom Java Transforms implementing your data manipulation.
Jolt Slide Deck : covers motivation, development, and transforms.
Javadoc explaining each transform DSL :
Running a Jolt transform means creating an instance of Chainr with a list of transforms.
The JSON spec for Chainr looks like : unit test.
The Java side looks like :
Chainr chainr = JsonUtils.classpathToList( "/path/to/chainr/spec.json" );
Object input = elasticSearchHit.getSource(); // ElasticSearch already returns hydrated JSon
Object output = chainr.transform( input );
return output;
The Shiftr transform generally does most of the "heavy lifting" in the transform chain. To see the Shiftr DSL in action, please look at our unit tests (shiftr tests) for nice bite sized transform examples, and read the extensive Shiftr javadoc.
Our unit tests follow the pattern :
{
"input": {
// sample input
},
"spec": {
// transform spec
},
"expected": {
// what the output of the transform looks like
}
}
We read in "input", apply the "spec", and Diffy it against the "expected".
To learn the Shiftr DSL, examine "input" and "output" json, get an understanding of how data is moving, and then look at the transform spec to see how it facilitates the transform.
For reference, this was the very first test we wrote.
There is a demo available at jolt-demo.appspot.com. You can paste in JSON input data and a Spec, and it will post the data to server and run the transform.
Note
Getting started code wise has its own doc.
If you can't get a transform working and you need help, create and Issue in Jolt (for now).
Make sure you include what your "input" is, and what you want your "output" to be.
Aside from writing your own custom code to do a transform, there are two general approaches to doing a JSON to JSON transforms in Java.
1) JSON -> XML -> XSLT or STX -> XML -> JSON
Aside from being a Rube Goldberg approach, XSLT is more complicated than Jolt because it is trying to do the whole transform with a single DSL.
2) Write a Template (Velocity, FreeMarker, etc) that take hydrated JSON input and write textual JSON output
With this approach you are working from the output format backwards to the input, which is complex for any non-trivial transform. Eg, the structure of your template will be dictated by the output JSON format, and you will end up coding a parallel tree walk of the input data and the output format in your template. Jolt works forward from the input data to the output format which is simpler, and it does the parallel tree walk for you.
Being in the Java JSON processing "space", here are some other interesting JSON manipulation tools to look at / consider :
The primary goal of Jolt was to improve "developer speed" by providing the ability to have a declarative rather than imperative transforms. That said, Jolt should have a better runtime than the alternatives listed above.
Work has been done to make the stock Jolt transforms fast:
Two things to be aware of :
Jolt Transforms and tools can be run from the command line. Command line interface doc here.
For the moment we have Cobertura configured in our poms.
mvn cobertura:cobertura
open jolt-core/target/site/cobertura/index.html
Currently, for the jolt-core artifact, code coverage is at 89% line, and 83% branch.