OutOfMemoryError when not preloading a schema

[szhem]

Consider we are parsing schema every time we'd like to validate a json object against it like the following:

String schema = "...";
String json = "...";

ObjectMapper mapper = new ObjectMapper();
JsonSchemaFactory schemaFactory = JsonSchemaFactory.byDefault();

for(int i = 0; i < 100000; i++) {
    JsonSchema jsonSchema = schemaFactory.getJsonSchema(mapper.readTree(schemaString));
    jsonSchema.validate(mapper.readTree(json));
}

Because of internally schema is generated with AnonymousSchemaKey, the CachingProcessor created within a schema factory caches every ProcessingResult of every validation within its cache. As a result - OutOfMemoryError is thrown when there are quite enough of such validation operations were performed.

It would be really great to have an option to prevent such kind of behaviour (i.e. disable caching) or in case of anonymous keys - just to disable caching at all, because it always leads to OutOfMemory errors.

[TheNilesh]

I was exactly running in this issue... And could not figure out the cause since a night. Moreover validator also creates thread pool for managing cache I guess, which isnt named appropriately, making it difficult to find the cause.

[TheNilesh]

Adding stacktrace for similar issue:

Exception in thread "main" com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2199)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
        at com.github.fge.jsonschema.core.processing.CachingProcessor.process(CachingProcessor.java:109)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:110)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:59)
        at com.github.fge.jsonschema.keyword.validator.draftv4.NotValidator.validate(NotValidator.java:57)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:129)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:59)
        at com.github.fge.jsonschema.keyword.validator.draftv4.AnyOfValidator.validate(AnyOfValidator.java:68)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:129)
        at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:56)
        at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:34)
        at com.github.fge.jsonschema.core.processing.ProcessingResult.of(ProcessingResult.java:79)
        at com.github.fge.jsonschema.main.JsonValidator.validate(JsonValidator.java:104)
        at com.github.fge.jsonschema.main.JsonValidator.validate(JsonValidator.java:124)
        at com.quickheal.protecti.uploader.utilities.jsonvalidator.QHJsonValidator.validate(QHJsonValidator.java:49)
        at com.quickheal.protecti.uploader.client.OperationClient.perform(OperationClient.java:82)
        at com.quickheal.protecti.uploader.client.UploaderClientApp.doThis(UploaderClientApp.java:372)
        at com.quickheal.protecti.uploader.client.UploaderClientApp.operation(UploaderClientApp.java:308)
        at com.quickheal.protecti.uploader.client.UploaderClientApp.main(UploaderClientApp.java:107)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at com.google.common.cache.LocalCache$EntryFactory$1.newEntry(LocalCache.java:450)
        at com.google.common.cache.LocalCache$EntryFactory.copyEntry(LocalCache.java:597)
        at com.google.common.cache.LocalCache$Segment.copyEntry(LocalCache.java:2149)
        at com.google.common.cache.LocalCache$Segment.expand(LocalCache.java:2949)
        at com.google.common.cache.LocalCache$Segment.storeLoadedValue(LocalCache.java:3117)
        at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2351)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
        at com.github.fge.jsonschema.core.processing.CachingProcessor.process(CachingProcessor.java:109)
        at com.github.fge.jsonschema.processors.validation.ValidationChain.process(ValidationChain.java:114)
        at com.github.fge.jsonschema.processors.validation.ValidationChain.process(ValidationChain.java:57)
        at com.github.fge.jsonschema.core.processing.ProcessorMap$Mapper.process(ProcessorMap.java:166)
        at com.github.fge.jsonschema.core.processing.ProcessingResult.of(ProcessingResult.java:79)
        at com.github.fge.jsonschema.core.processing.CachingProcessor$1.load(CachingProcessor.java:128)
        at com.github.fge.jsonschema.core.processing.CachingProcessor$1.load(CachingProcessor.java:120)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
        at com.github.fge.jsonschema.core.processing.CachingProcessor.process(CachingProcessor.java:109)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:110)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:59)
        at com.github.fge.jsonschema.keyword.validator.draftv4.NotValidator.validate(NotValidator.java:57)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:129)
        at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:59)

[neuzrui]

I am running into the same issue too!

I have similar stacktrace TheNilesh posted and I took a heap dump on my server and the suspects that Eclipse MAT reported were:

• 1,303,277 instances of "com.fasterxml.jackson.databind.node.ObjectNode", loaded by "sun.misc.Launcher$AppClassLoader @ 0x832098f0" occupy 450,932,352 (27.75%) bytes.

• 8 instances of "com.google.common.cache.LocalCache$Segment", loaded by "sun.misc.Launcher$AppClassLoader @ 0x832098f0" occupy 370,108,312 (22.77%) bytes.

• 1,303,277 instances of "com.github.fge.jsonschema.core.tree.CanonicalSchemaTree", loaded by "sun.misc.Launcher$AppClassLoader @ 0x832098f0" occupy 260,436,032 (16.02%) bytes.

• 3 instances of "com.google.common.cache.LocalCache", loaded by "sun.misc.Launcher$AppClassLoader @ 0x832098f0" occupy 221,334,584 (13.62%) bytes.

Seems like they are all objects involved in the json validation process.

[huggsboson]

@neuzrui yours sounds like another instance of #135. It should be fixed in 2.2.8.