quick-perf / quickperf

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties
https://github.com/quick-perf/doc/wiki/QuickPerf
Apache License 2.0
465 stars 67 forks source link

Customize or reduce assertion errors description #214

Open MelleD opened 7 months ago

MelleD commented 7 months ago

I would like to customize or reduce the error messages.

With batch operations, the output when an error occurs is far too large and confusing with the entire SQL in it. It also makes the tests much slower.

For example, this appears 100 times in the log and in the test case and clutters up the log.

Time:8, Success:True, Type:Prepared, Batch:True, QuerySize:1, BatchSize:100, Query:["update [http_endpoint] set [http_endpoint].[model_reference] = ?, [http_endpoint].[last_modified_by] = ?, [http_endpoint].[last_modified_at] = ? where [http_endpoint].[aspect_id] = ?"], Params:[(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT)),(foo,bar,1702550385629,NULL(BIGINT))]

jeanbisutti commented 7 months ago

@MelleD You can use -DlimitQuickPerfSqlInfoOnConsole=true to limit the console verbosity (thanks @kanedafromparis for having implemented this!). It will apply to all the tests. To limit the console verbosity to only one test, you can use the property with the JvmOptions annotation applied to the test method.

Perhaps the best option for JDBC batches with an unreduced data set is to only use the ExpectJdbcBatching annotation.

Does this help you?

MelleD commented 7 months ago

@jeanbisutti

At first I didn't see any difference. I'll set it to the base class so that it's for all tests. Are the options described somewhere?

@JvmOptions( "-DlimitQuickPerfSqlInfoOnConsole=true" ) and for spring JvmAnnotationBuilder.jvmOptions( "limitQuickPerfSqlInfoOnConsole=true" )

Tried both with -D and without.

Maybe here is the issue, the property is not in the System.getProperty

   public static final SystemProperty<Boolean> SIMPLIFIED_SQL_DISPLAY =
            new SystemProperty<Boolean>() {

                private final String name = "limitQuickPerfSqlInfoOnConsole";

                @Override
                public Boolean evaluate() {
                    String booleanAsString = System.getProperty(name);
                    return Boolean.valueOf(booleanAsString);
                }

                @Override
                public String buildForJvm(String propertyValue) {
                    return "-D" + name + "=" + propertyValue;
                }
            };
jeanbisutti commented 7 months ago

@MelleD Sorry, it can't work with JvmOption (a JVM property annotation should be added to QuickPerf).

You could set the property from your build tool (see Surfire with Maven).

You could also programmatically use System.setProperty("limitQuickPerfSqlInfoOnConsole", true) / System.setProperty("limitQuickPerfSqlInfoOnConsole", false).

With batch operations, you could also only use the ExpectJdbcBatching annotation.

MelleD commented 7 months ago

You could also programmatically use System.setProperty("limitQuickPerfSqlInfoOnConsole", true) / System.setProperty("limitQuickPerfSqlInfoOnConsole", false) .

Yes that's clear, but not really recommended in Spring (tests) :) Normally you use the properties/yaml file, the TestSourceProperty or in SpringBootTest the properties key.

Sorry, it can't work with JvmOption (a JVM property annotation should be added to QuickPerf).

Do you think it is possible to extend that we can use JvmOption here?

For me I would use this in my QuickPerfConfig JvmAnnotationBuilder.jvmOptions( "limitQuickPerfSqlInfoOnConsole=true" ) or also good a spring boot properties in application-test.properties or yaml

You could set the property from your build tool (see Surfire with Maven)

That's clear but I would avoid this, also not so nice for local development to set always the property

jeanbisutti commented 7 months ago

Do you think it is possible to extend that we can use JvmOption here?

A JvmSystemPropertry annotation should be added. I can help you to do it and contribute if you are interested.

You are using Spring. So, you could try to set the property from Spring, for example: @SpringBootTest(properties = { "limitQuickPerfSqlInfoOnConsole=true" }).

That's clear but I would avoid this, also not so nice for local development to set always the property

You could enable the property from Surfire. And for local development, investigation, use @DisplaySqlQuery.

MelleD commented 7 months ago

A JvmSystemPropertry annotation should be added. I can help you to do it and contribute if you are interested.

That sounds awesome

You are using Spring. So, you could try to set the property from Spring, for example: @SpringBootTest(properties = { "limitQuickPerfSqlInfoOnConsole=true" }).

This was my preferred way also tried the properties file. But both did not work.

jeanbisutti commented 7 months ago

@MelleD Another possibility would be to be provide a way to configure some annotations with a short error display (configuration at annotation level). It will avoid the creation of a new JVM. What do you prefer? I can help you to contribute in both cases if you are interested.