bazaarvoice / jolt

JSON to JSON transformation library written in Java.
Apache License 2.0
1.54k stars 328 forks source link

Make flat json array from nested one via jolt #1139

Closed rustoherox closed 2 years ago

rustoherox commented 2 years ago

Hi everyone!

I'm trying to flat array and it view as simple case, but I can't reach the goal.

Here is simple json input:

{
  "MainArray": {
    "mattr1": "some_value1",
    "mattr2": "some_value2",
    "Subarray": {
         "sattr1": 1,
         "sattr2": 2,
         "sattr3": 3
    }
  }
}

Then I would like to convert it to array of arrays [key, value], Here is Desired Output:

{
  "ResultArray" : [ [ "mattr1", "some_value1" ], [ "mattr2", "some_value2" ], [ "sattr1", 1 ], [ "sattr2", 2 ], [ "sattr1", 3 ] ]
}

Here is my spec:

[
  {
    "operation": "shift",
    "spec": {
      "MainArray": {
        "*": {
            "$": "ResultArray[#2][0]",
            "@": "ResultArray[#2][1]"
        },
        "Subarray": {
          "*": {
              "$": "ResultArray[#3][#2][0]",
              "@": "ResultArray[#3][#2][1]"
          }
        }
      }
    }
  }
]

And I don't understand how to fix problem when Subarrays is inside extra array in my result:

{
  "ResultArray" : [ [ "mattr1", "some_value1" ], [ "mattr2", "some_value2" ], [ [ "sattr1", 1 ], [ "sattr2", 2 ], [ "sattr3", 3 ] ] ]
}

Any advise, please.

ouxuyong commented 2 years ago

@rustoherox This spec of mine can achieve the effect you want, I hope it can help you. In addition, there is a jolt tutorial project, welcome to communicate and discuss jolt-universe.

[
  {
    "operation": "shift",
    "spec": {
      "MainArray": {
        "*": {
          "$": "ResultArrayTemp0[#2]",
          "@": "ResultArrayTemp0[#2]"
        },
        "Subarray": {
          "*": {
            "$": "ResultArrayTemp1[#2]",
            "@": "ResultArrayTemp1[#2]"
          }
        }
      }
    }
  }, {
    "operation": "shift",
    "spec": {
      "ResultArrayTemp0": {
        "*": "ResultArray[]"
      },
      "ResultArrayTemp1": {
        "*": "ResultArray[]"
      }
    }
  }
]
rustoherox commented 2 years ago

@ouxuyong, Thanks for variant! Is it possible in theory do similar in one shift operation? It looks like it is...

ouxuyong commented 2 years ago

@rustoherox It's theoretically possible, but I just tried it and it doesn't seem to work, I'll look at the source code later

ouxuyong commented 2 years ago

@rustoherox The array subscript of "ResultArray" is determined by the value of the "hashCount" field of "MainArray". If "hashCount" wants to add 1 to x, the "Subarray" must be traversed, so according to the source code, a layer of "spec" configuration The effect you want cannot be achieved.
image

rustoherox commented 2 years ago

@ouxuyong, Thank you very much for detailed description, I close issue and will use append subarray to mainarray. Best wishes.