search

IBMStreams / streamsx.json

Toolkit for working with JSON in SPL applications.
http://ibmstreams.github.io/streamsx.json/
Other
3 stars 19 forks source link

TupleToJson should flatten Json output if a spl list contains only one element #43

Closed chanskw closed 4 years ago

chanskw commented 8 years ago

Assuming a tuple has the following type:

type DataType = tuple<rstring a,list<rstring> b,list<rstring> c>;

If a tuple of this data type contains the following: Data1: data={a="A", b=["B"], c=["C"]}

The following will get generated by the TupleToJSON operator: {"a":"A","b":["B"],"c":["C"]}

This is cumbersome to parse by downstream applications. User would like to see the following generated instead:

{"a":"A","b":"B","c":"C"}

ddebrunner commented 8 years ago

Since it's standard valid JSON, why would it be cumbersome to parse? Especially as the dwonstream application has to handle multiple values in the list which requires the same parsing.

chanskw commented 8 years ago

This is a requirement I received from talking to an external user. I have to check what they are doing downstream. But they do not like having these optional elements embedded. I think in their case, these elements can have one or zero value.

ddebrunner commented 8 years ago

Seems strange that the SPL schema is a list then rather than a single value.

Just seems this special cases have the potential to make the operator become complicated and thus harder for users to understand.

m-kotowski commented 8 years ago

The customer wants to suppress optional elements in the JSON output if the source value is not present. Since SPL does not support optional values, the workaround is either to use additional attributes (e.g. boolean) to indicate that the value is present or absent, or to use a list, which can have 0 or 1 elements.

From my point of view, this issue asks for a non-automatic JSON schema. Today, the JSON schema is derived from the tuple schema. To solve the customer problem, a customer-provided schema would be helpful. This schema would specify an attribute as optional and how to decide whether the value is present or absent.

Example 1 (boolean): JSON "b" is optional, JSON "b" is present if the SPL boolean isBPresent attribute is true, use SPL b value

Example 2 (list): "b" is optional, "b" is present if the size of the SPL b list equals 1, use SPL b[0] value

rnostream commented 4 years ago

Old issue, it was about workaround for optional elements which we have meanwhile in Streams and which are supported in JSON toolkit