search

bazaarvoice / jolt

JSON to JSON transformation library written in Java.
Apache License 2.0
1.54k stars 328 forks source link

Jolt transformer throwing OutOfMemoryError Exception #1205

Open purushottambaghel opened 1 year ago

purushottambaghel commented 1 year ago

Whenever I am running this spec and input, jolt tranformer is getting stuck. It is neither getting timeout nor sending any exception.

Screenshot 2023-06-30 at 5 08 30 PM

Input

{
  "payload": {
    "style_properties": {
      "style_fields": [
        {
          "name": "Rishi",
          "label": "Rishi multi",
          "type": "multi_select",
          "value": [
            "banita",
            "ajith"
          ]
        }
      ]
    }
  }
}

spec

[
    {
        "operation": "shift",
        "spec": {
            "payload": {
                "style_properties": {
                    "display_id": "payload[0].object.display_id",
                    "style_fields": {
                        "*": {
                            "value": {
                                "*": {
                                    "@1": "payload[0].content.changes"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
]

I have tried running this with v0.1.6 on my local. I am getting OutOfMemoryError. Can someone please help.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:3210)
        at java.util.Arrays.copyOf(Arrays.java:3181)
        at java.util.ArrayList.grow(ArrayList.java:267)
        at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:241)
        at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:233)
        at java.util.ArrayList.add(ArrayList.java:464)
        at com.bazaarvoice.jolt.shiftr.ShiftrTraversr.handleFinalSet(ShiftrTraversr.java:55)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:54)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:67)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:67)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:67)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:67)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:67)
        at com.bazaarvoice.jolt.traversr.traversal.BaseTraversalStep.traverse(BaseTraversalStep.java:67)
        at com.bazaarvoice.jolt.traversr.Traversr.set(Traversr.java:148)
        at com.bazaarvoice.jolt.common.PathEvaluatingTraversal.write(PathEvaluatingTraversal.java:101)
        at com.bazaarvoice.jolt.shiftr.spec.ShiftrLeafSpec.apply(ShiftrLeafSpec.java:139)
        at com.bazaarvoice.jolt.shiftr.spec.ShiftrCompositeSpec.apply(ShiftrCompositeSpec.java:218)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.applyKeyToComputed(ExecutionStrategy.java:327)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.applyKeyToLiteralAndComputed(ExecutionStrategy.java:317)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.access$200(ExecutionStrategy.java:26)
        at com.bazaarvoice.jolt.common.ExecutionStrategy$4.processList(ExecutionStrategy.java:222)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.process(ExecutionStrategy.java:284)
        at com.bazaarvoice.jolt.shiftr.spec.ShiftrCompositeSpec.apply(ShiftrCompositeSpec.java:222)
        at com.bazaarvoice.jolt.common.ExecutionStrategy$1.processMap(ExecutionStrategy.java:44)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.process(ExecutionStrategy.java:281)
        at com.bazaarvoice.jolt.shiftr.spec.ShiftrCompositeSpec.apply(ShiftrCompositeSpec.java:222)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.applyKeyToComputed(ExecutionStrategy.java:327)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.access$100(ExecutionStrategy.java:26)
        at com.bazaarvoice.jolt.common.ExecutionStrategy$3.processList(ExecutionStrategy.java:183)
        at com.bazaarvoice.jolt.common.ExecutionStrategy.process(ExecutionStrategy.java:284)
        at com.bazaarvoice.jolt.shiftr.spec.ShiftrCompositeSpec.apply(ShiftrCompositeSpec.java:222)