Significant performance decrease

[eloo]

Hi,

we are using your library in one of our projects to validate json (what surprise). We have used 0.12.0 for a while and now we have updated to 2.1.0 (just exchanged the versions) and we noticed a huge performance decrease now.

With 0.12.0
Mean requests per second: 29.508196
Successful: 1800 / 1800
Max: 731 ms
Mean: 183 ms

With 2.1.0
Mean requests per second: 15.0
Successful: 532 / 1800
Max: 60254 ms
Mean: 52357 ms

We see also that the CPU usage during our performance test is nearly doubled with the 2.1.0 version.

We have not yet investigated whats the root cause.. but we have reverted the version to 0.17.0 and now the performance is normal again.

So are there any known issues in 2.1.0 related to the performance? Is this maybe fixed in 3.0.0 rc?

FYI: The metrics you see are created during a simple GET request performance test. During this request the json will be validated when the data is loaded from the database.

Thanks eloo

[leadpony]

Hello @eloo Thank you for contacting me. I am not sure immediately what makes the difference in your performance test. Your report shows me that the success rate is not 100% with 2.1.0 and I believe that there is performance penalty if the input JSON is not valid against the schema, because the library must do error handling. Could you tell me why the success rate get lower value with 2.1.0 and what type of validation failures is happening in your test?

[eloo]

Hi, yes there is maybe additional handling if the schema is not valid. But i guess the main reason for the lower success rate is that timeouts are reached. As you see the mean around 53seconds instead of 183ms.

But you got a point a should check the logs if maybe now the json validation is failing. Which would also lead to an direct relation to the version bump.

[eloo]

Okay i've just digged a bit deeper and found the following:

• Looks like the JsonValidationService.newInstance(); call is taking lower in the later version
• We have created a new service instance everytime we are going to validate a json

So is it safe to reuse the jsonvalidationservice instances?

Some number: Our method which is causing the newInstance call has the following metrics. 1.0.0: count: 1325.0 total time: 34.328442577 s

0.17.0: count: 1325.0 total time: 6.436005346

So it looks like there was a major change with has effected the performance.

For us its fine if we could simple reuse the jsonvalidationservice

Thanks

[leadpony]

Yes, the instance of JsonValidationService can be reused and should be so. Please see https://www.javadoc.io/doc/org.leadpony.justify/justify/2.1.0/org.leadpony.justify/org/leadpony/justify/api/JsonValidationService.html

All the methods in this class are safe for use by multiple concurrent threads. For most use-cases, only one instance of JsonValidationService is required within the application.

Furthermore if the input JSON documents will be validated against a JSON schema, the schema should be loaded once at the application startup , which will reduce the response time much more.

[eloo]

thanks for the information. IMHO we can close this issue as it was mostly misusage.

[leadpony]

OK, I close this problem. Thank you for your report.