IBMStreams / streamsx.json

Toolkit for working with JSON in SPL applications.
http://ibmstreams.github.io/streamsx.json/
Other
3 stars 19 forks source link

Add a flatten array operator #35

Closed ddebrunner closed 4 years ago

ddebrunner commented 9 years ago

Some data sources provide a single JSON object containing an array with multiple values that might be better handled as individual tuples, e.g. SFPark data returns a JSONObject containing some meta data and then an array of parking locations.

Something like:

steam<Json> F = FlattenJSONArray(I) { params array: "locations"; // attribute to be flattened metadata: "timestamp", "success"; // optional list of meta-attributes to copy into each tuple }

hildrum commented 9 years ago

Any reason why an operator rather than a function?

ddebrunner commented 9 years ago

I hadn't thought about an function, but that would require the flattening to be done by every caller in a Custom operator.

The operator would submit N tuples for each input tuple, where N is the size of the array in the JSON object represented by the input tuple.

hildrum commented 9 years ago

I just realized you can't do this with a java function, so now I think an operator is the best approach here.