JKISoftware / JKI-JSON-Serialization

JSON Serialization & Deserialization Library for LabVIEW
http://jki.net/tools#json
BSD 3-Clause "New" or "Revised" License
25 stars 10 forks source link

Multi dimestional array not implemented #5

Closed 3esmit closed 8 years ago

3esmit commented 8 years ago

I'm currently developing a Variant/BSON (to work with MongoDB) library, and I find really intersting to see how JKI would accomplish this.
I'm copying the drjdpowell Variant/JSON library, that seems to work well with any dimension size and array element type.
I tried to change this repo code to make Multi D support but using OpenG 'Array to Array of VData__ogtk.vi' makes things difficult, as it reshapes to a 1D array. image I could make it work using the example of drjdpowell, but I'm not sure if this is the best way to do it.

Consider a VI 'Variant to Array of Variant.vi' that inputs 'Variant' and returns 'Array'; When the input is like Array it will return a Array normally, but in the case of input be Array (MultiD array) it would return Array. So when the reentrant Flatten to JSON.vi is called, the Variant passed would be an Array.
This way could even work for 1D or any D, but maybe the primitive flatten is optimized enough to make a each type case and sacrefice memory to processing, that might be important if someone may be using this for some real time acquisiton with 1D arrays.
The same idea could be applied with 2D arrays that are simple and very common to be used in this cases, like 2D array of numerics, but of course not for arrays of clusters.

tmaila commented 8 years ago

Right now we don't support multidimensional arrays as they are not supported natively in JSON. Before we can support multidimensional arrays, we need to decide what is the expected behavior in flattening and unflattening such arrays.

In JSON you can create arrays of arrays that would be the closest item to multidimensional arrays in LabVIEW. For example the following would be a valid JSON object:

[ [ ], [1], [1, 2], [1, 2, 3, 4, 5, 6, 7], [[1,2],[1,2]]]

Such arrays can have rows and columns that vary in size from element to element. Furthermore the dimensionality can increase for some rows and columns. LabVIEW does not natively support this kind of data structures as multidimensional arrays.

What LabVIEW does support is arrays of clusters of arrays which would support variable sized array elements but using them raises a question how to flatten them. The natural flattening would be something like the following:

[ { [ 1 ] }, { [ 1, 2 ] } ]

But that may not be what the user expects or wants. Also clusters of arrays doesn't really solve the problem of variable dimensionality.

To include support for multidimensional arrays to the JSON library we need to come up with optimal and most natural expected behavior for both flattening and unflattening operations. Only after that can we focus solving the technical issues of how to exactly implement that. But I don't think the technical solution is a road block here.

3esmit commented 8 years ago

I would expect that JSON Serialization input would accept "anything" and returns all as variants.

Example [ [ ], ["1"], [1, 2], [1, 2, 3, 4, 5, 6, 7], [["a","b"],{[1,2]}]] It does not matter the input. If the type was expecting an integer the searlization would do it's best to convert whatever data is there to the type defined. So "1" would be converted to 1. The same if type expected was String, so the output would be converted to String, and in case it's a document, it would be a toString of the nested document. If the format is not convertable, like type expected is number and field contains a document, then an error or warning would be throwed.

Flattening would be limited to LV data structure, unless a variant's array to be supplied with the mixed data type. I currently do this in my MongoDB library and works well: image

jimkring commented 8 years ago

I have thoughts on this, too -- this is an important feature request.

Right now, I really need to be able to flatten and unflatten a 2D array of strings for Google Sheets access. For example:

{
  "range": "Sheet1!A1:E1",
  "majorDimension": "ROWS",
  "values": [
    [
      "1",
      "2",
      "3",
      "4",
      "5"
    ]
  ]
}

In the case of flattening a 2D array from LabVIEW to JSON, it's a no brainer since each LabVIEW rows/columns have to all be the same length in a multi-dimensional array.

In the case of unflattening it's trickier and the options are to either (A) truncate all LabVIEW rows to the shortest row/column, or (B) pad all rows/columns (in LabVIEW with default data elements) to the longest row/column.

Padding the LabVIEW array with default data seems to be the best option, since there is no loss of data. I can see cases where it would be nice to be able to unflatten each JSON row/column separately and I have other feature ideas on how to do that -- specifically, we need to be able to tag LabVIEW strings as #JSON similar to how EasyXML has an #xml tag.

tmaila commented 8 years ago

Feature added to release 1.0.6