simdjson / simdjson-java

A Java version of simdjson, a high-performance JSON parser utilizing SIMD instructions
Apache License 2.0
240 stars 17 forks source link

Simdjson cannot compress structures #39

Open heykirby opened 5 months ago

heykirby commented 5 months ago

Simdjson cannot compress structures for example json { "field1": { "field2": "xx" } } I cannot get the compressed value for field1, value is { "field2": "xx" }

piotrrzysko commented 4 months ago

Do I understand correctly that you would like to read the value of field1 as a string? Something like this:

JsonValue json = parser.parse(...);
Iterator<Map.Entry<String, JsonValue>> it = json.objectIterator();
Map.Entry<String, JsonValue> field1 = it.next();
System.out.println(field1.getValue().asString()); // this prints '{ "field2": "xx" }'
heykirby commented 4 months ago

yes, it looks like simdjson has implemented this capability,thanks

andyglow commented 4 months ago

for me field1.getValue().asString() doesn't work if the value is either <array> or <object> gets me something like this instead

Caused by: java.lang.StringIndexOutOfBoundsException: Range [3198, 3198 + 1952541807) out of bounds for length 35651584
         at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:55)
         at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:52)
         at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
         at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
         at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
         at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromIndexSize(Preconditions.java:118)
         at java.base/jdk.internal.util.Preconditions.checkFromIndexSize(Preconditions.java:397)
         at java.base/java.lang.String.checkBoundsOffCount(String.java:4849)
         at java.base/java.lang.String.<init>(String.java:522)
         at org.simdjson.JsonValue.getString(JsonValue.java:87)
         at org.simdjson.JsonValue.asString(JsonValue.java:81)
piotrrzysko commented 4 months ago

The snippet I provided in the previous comment was just to verify if I understand the issue correctly. I'm not saying that currently this line works according to the comment:

System.out.println(field1.getValue().asString()); // this prints '{ "field2": "xx" }'
andyglow commented 4 months ago

gotcha. are there plans to make asString handle these scenarios, @piotrrzysko ? in some cases it would be great to be able to reuse original slice of buffer to stringify the value instead of building a new one traversing the value

piotrrzysko commented 4 months ago

I think we can try to add it. However, I'm currently busy working on https://github.com/simdjson/simdjson-java/issues/35, so I can't promise when I'll be able to do that.

andyglow commented 4 months ago

I see. That would be wonderful if you could make it happen at some point. I'll be waiting for it :)