search

hcoles / pitest

State of the art mutation testing system for the JVM
Apache License 2.0
1.7k stars 357 forks source link

Is it possible to use the pitest tool with the jqwik tool (property-based test tool)? #878

Open ricardosm opened 3 years ago

ricardosm commented 3 years ago

Can I use the jqwik tool (property-based test tool) with the pitest?

My goal is to automatically generate the values (inputs) for my test cases and evaluate the strength of these values against the mutants generated by Pitest.

The problem is that running Pitest 1.6.4 with jqwik 1.5.0 the Pitest tool did not recognize the properties (@Property) in my PropertiesTest classes.

Below is an example (useless example):

public class Word {

      private String string;

      public Word(String string) {
          this.string = string; 
      }

      public int getSize() {
          return string.length();
      }
}
public class WordPropertiesTest {

      @Property 
      void concatenationLength(@ForAll String s1, @ForAll String s2) {

          Word string1 = new Word(s1);
          Word string2 = new Word(s2);

          Word string3 = new Word(s1 + s2);

          Assertions.assertEquals(string1.getSize() + string2.getSize(), string3.getSize());
      }
}

Below is the result of the Pitest execution (the pitest tool does not find the @Property test case):

pit: [pitest] 22:11:13 PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue. [pitest] 22:11:13 PIT >> INFO : Sending 2 test classes to minion [pitest] 22:11:13 PIT >> INFO : Sent tests to minion [pitest] 22:11:13 PIT >> INFO : MINION : 22:11:13 PIT >> INFO : Checking environment [pitest] 22:11:13 PIT >> INFO : MINION : 22:11:13 PIT >> INFO : Found 0 tests [pitest] 22:11:13 PIT >> INFO : MINION : 22:11:13 PIT >> INFO : Dependency analysis reduced number of potential tests by 0 [pitest] 22:11:13 PIT >> INFO : MINION : 22:11:13 PIT >> INFO : 0 tests received [pitest] 22:11:13 PIT >> INFO : Calculated coverage in 0 seconds. [pitest] 22:11:13 PIT >> INFO : Incremental analysis reduced number of mutations by 0 [pitest] 22:11:13 PIT >> INFO : Created 1 mutation test units [pitest] 22:11:13 PIT >> INFO : Completed in 0 seconds [pitest]/================================================================================ [pitest] - Mutators [pitest] ================================================================================ [pitest] > org.pitest.mutationtest.engine.gregor.mutators.experimental.MemberVariableMutator [pitest] >> Generated 1 Killed 0 (0%) [pitest] > KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 [pitest] > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 [pitest] > NO_COVERAGE 1 [pitest] -------------------------------------------------------------------------------- [pitest] > org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator [pitest] >> Generated 1 Killed 0 (0%) [pitest] > KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 [pitest] > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 [pitest] > NO_COVERAGE 1 [pitest] -------------------------------------------------------------------------------- [pitest] > org.pitest.mutationtest.engine.gregor.mutators.PrimitiveReturnsMutator [pitest] >> Generated 1 Killed 0 (0%) [pitest] > KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 [pitest] > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 [pitest] > NO_COVERAGE 1 [pitest] -------------------------------------------------------------------------------- [pitest] > org.pitest.mutationtest.engine.gregor.mutators.NonVoidMethodCallMutator [pitest] >> Generated 1 Killed 0 (0%) [pitest] > KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 [pitest] > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 [pitest] > NO_COVERAGE 1 [pitest] -------------------------------------------------------------------------------- [pitest] ================================================================================ [pitest] - Timings [pitest] ================================================================================ [pitest] > scan classpath : < 1 second [pitest] > coverage and dependency analysis : < 1 second [pitest] > build mutation tests : < 1 second [pitest] > run mutation analysis : < 1 second [pitest] -------------------------------------------------------------------------------- [pitest] > Total : < 1 second [pitest] -------------------------------------------------------------------------------- [pitest] ================================================================================ [pitest] - Statistics [pitest] ================================================================================ [pitest] >> Generated 4 mutations Killed 0 (0%) [pitest] >> Mutations with no coverage 4. Test strength 0% [pitest] >> Ran 0 tests (0 tests per mutation)

BUILD SUCCESSFUL Total time: 2 seconds

Arraying commented 2 years ago

Facing the exact same issue. Did you/anyone else find a solution?

hcoles commented 2 years ago

I'm assuming jqwik is built on junit5. Is the pitest junit 5 plugin configured?

Arraying commented 2 years ago

Yup, it's on the platform. I'm not sure what you mean by configured, but when we mix in "regular" @Test units from JUnit5 then these get picked up.

As far as I can tell, everything is in the classpath. Here's a verbose dump:

22:45:55 PIT >> INFO : ---------------------------------------------------------------------------
22:45:55 PIT >> INFO : Enabled (+) and disabled (-) features.
22:45:55 PIT >> INFO : -----------------------------------------
22:45:55 PIT >> INFO : +FANN           Filters mutations in classes and methods with matching annotations of class or runtime retention
22:45:55 PIT >> INFO :   [annotation]    Annotation to avoid (full package name not required)
22:45:55 PIT >> INFO : +FENUM          Filters mutations in enum constructors
22:45:55 PIT >> INFO : +FFBLOCK        Filters mutations in code duplicated by finally block inlining
22:45:55 PIT >> INFO : +FFEACH         Filters mutations in compiler generated code that implements for each loops
22:45:55 PIT >> INFO : +FFLOOP         Filters any mutations to increments in for loops as they may cause timeouts
22:45:55 PIT >> INFO : +FINFINC        Filters mutations to increments that may cause infinite loops
22:45:55 PIT >> INFO : +FINFIT         Filters mutations that may cause infinite loops by removing calls to iterator.next
22:45:55 PIT >> INFO : +FINULL         Filters mutations in compiler generated code that checks for null by calling getClass
22:45:55 PIT >> INFO : +FKOTLIN        Filters out junk mutations in bytecode created by compiler for kotlin language features
22:45:55 PIT >> INFO : +FLOGCALL       Filters mutations in code that makes calls to logging frameworks
22:45:55 PIT >> INFO : +FMRNULL        Filters mutations in compiler generated code that inserts Objects.requireNonNull for method references
22:45:55 PIT >> INFO : +FRETEQUIV      Filters return vals mutants with bytecode equivalent to the unmutated class
22:45:55 PIT >> INFO : +FSEQUIVEQUALS  Filters equivalent mutations that affect only performance in short cutting equals methods
22:45:55 PIT >> INFO : +FSTATI         Filters mutations in static initializers and code called only from them
22:45:55 PIT >> INFO : +FSTATINIT      Filters mutations in static initializers and code called only from them
22:45:55 PIT >> INFO : +FTRYWR         Filters mutations in code generated for try with resources statements
22:45:55 PIT >> INFO : +_internal_activate_by_output_string
22:45:55 PIT >> INFO : -CLASSLIMIT     Limits the maximum number of mutations per class
22:45:55 PIT >> INFO :   [limit]         Integer value for maximum mutations to create per class
22:45:55 PIT >> INFO : -EXPORT         Exports mutants bytecode and other details to disk
22:45:55 PIT >> INFO : ---------------------------------------------------------------------------
22:45:55 PIT >> FINE : Running report with ReportOptions [targetClasses=[delft.MathArrays], excludedMethods=[], excludedClasses=[], excludedTestClasses=[], codePaths=[], reportDir=C:\Users\User\Programming\Uni\assignments\property-based-testing\unique\output\pitest, historyInputLocation=null, historyOutputLocation=null, sourceDirs=[C:\Users\User\AppData\Local\Temp\1637358352212-0], classPathElements=[C:\Users\User\AppData\Local\Temp\1637358352212-0, C:\Users\User\Programming\Uni\assignments\property-based-testing\unique\target\classes, C:\Users\User\.m2\repository\io\github\cse1110\andy\0.27-SNAPSHOT\andy-0.27-SNAPSHOT.jar, C:\Users\User\.m2\repository\org\slf4j\slf4j-log4j12\1.7.30\slf4j-log4j12-1.7.30.jar, C:\Users\User\.m2\repository\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar, C:\Users\User\.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar, C:\Users\User\.m2\repository\org\assertj\assertj-core\3.15.0\assertj-core-3.15.0.jar, C:\Users\User\.m2\repository\net\jqwik\jqwik\1.5.1\jqwik-1.5.1.jar, C:\Users\User\.m2\repository\org\apiguardian\apiguardian-api\1.1.1\apiguardian-api-1.1.1.jar, C:\Users\User\.m2\repository\net\jqwik\jqwik-api\1.5.1\jqwik-api-1.5.1.jar, C:\Users\User\.m2\repository\org\opentest4j\opentest4j\1.2.0\opentest4j-1.2.0.jar, C:\Users\User\.m2\repository\org\junit\platform\junit-platform-commons\1.7.1\junit-platform-commons-1.7.1.jar, C:\Users\User\.m2\repository\net\jqwik\jqwik-web\1.5.1\jqwik-web-1.5.1.jar, C:\Users\User\.m2\repository\net\jqwik\jqwik-time\1.5.1\jqwik-time-1.5.1.jar, C:\Users\User\.m2\repository\org\mockito\mockito-core\3.10.0\mockito-core-3.10.0.jar, C:\Users\User\.m2\repository\net\bytebuddy\byte-buddy\1.11.0\byte-buddy-1.11.0.jar, C:\Users\User\.m2\repository\net\bytebuddy\byte-buddy-agent\1.11.0\byte-buddy-agent-1.11.0.jar, C:\Users\User\.m2\repository\org\objenesis\objenesis\3.2\objenesis-3.2.jar, C:\Users\User\.m2\repository\org\junit\jupiter\junit-jupiter-engine\5.7.0\junit-jupiter-engine-5.7.0.jar, C:\Users\User\.m2\repository\org\junit\platform\junit-platform-engine\1.7.0\junit-platform-engine-1.7.0.jar, C:\Users\User\.m2\repository\org\junit\jupiter\junit-jupiter-api\5.7.0\junit-jupiter-api-5.7.0.jar, C:\Users\User\.m2\repository\org\junit\jupiter\junit-jupiter-params\5.7.0\junit-jupiter-params-5.7.0.jar, C:\Users\User\.m2\repository\org\junit\platform\junit-platform-launcher\1.7.0\junit-platform-launcher-1.7.0.jar, C:\Users\User\.m2\repository\org\pitest\pitest\1.6.7\pitest-1.6.7.jar, C:\Users\User\.m2\repository\org\pitest\pitest-junit5-plugin\0.14\pitest-junit5-plugin-0.14.jar, C:\Users\User\.m2\repository\org\pitest\pitest-command-line\1.6.7\pitest-command-line-1.6.7.jar, C:\Users\User\.m2\repository\org\pitest\pitest-entry\1.6.7\pitest-entry-1.6.7.jar, C:\Users\User\.m2\repository\org\jacoco\org.jacoco.core\0.8.7\org.jacoco.core-0.8.7.jar, C:\Users\User\.m2\repository\org\ow2\asm\asm\9.1\asm-9.1.jar, C:\Users\User\.m2\repository\org\ow2\asm\asm-commons\9.1\asm-commons-9.1.jar, C:\Users\User\.m2\repository\org\ow2\asm\asm-analysis\9.1\asm-analysis-9.1.jar, C:\Users\User\.m2\repository\org\ow2\asm\asm-tree\9.1\asm-tree-9.1.jar, C:\Users\User\.m2\repository\org\jacoco\org.jacoco.report\0.8.7\org.jacoco.report-0.8.7.jar, C:\Users\User\.m2\repository\org\eclipse\jdt\org.eclipse.jdt.core\3.25.0\org.eclipse.jdt.core-3.25.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.resources\3.15.100\org.eclipse.core.resources-3.15.100.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.expressions\3.8.0\org.eclipse.core.expressions-3.8.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.runtime\3.23.0\org.eclipse.core.runtime-3.23.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.osgi\3.17.0\org.eclipse.osgi-3.17.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.equinox.common\3.15.0\org.eclipse.equinox.common-3.15.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.jobs\3.12.0\org.eclipse.core.jobs-3.12.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.equinox.registry\3.11.0\org.eclipse.equinox.registry-3.11.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.equinox.preferences\3.9.0\org.eclipse.equinox.preferences-3.9.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.contenttype\3.8.0\org.eclipse.core.contenttype-3.8.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.equinox.app\1.6.0\org.eclipse.equinox.app-1.6.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.filesystem\1.9.100\org.eclipse.core.filesystem-1.9.100.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.text\3.12.0\org.eclipse.text-3.12.0.jar, C:\Users\User\.m2\repository\org\eclipse\platform\org.eclipse.core.commands\3.10.100\org.eclipse.core.commands-3.10.100.jar, C:\Users\User\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.12.3\jackson-core-2.12.3.jar, C:\Users\User\.m2\repository\org\apache\commons\commons-lang3\3.12.0\commons-lang3-3.12.0.jar, C:\Users\User\.m2\repository\commons-io\commons-io\2.11.0\commons-io-2.11.0.jar, C:\Users\User\.m2\repository\com\google\code\gson\gson\2.8.8\gson-2.8.8.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-java\4.0.0\selenium-java-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-api\4.0.0\selenium-api-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-chrome-driver\4.0.0\selenium-chrome-driver-4.0.0.jar, C:\Users\User\.m2\repository\com\google\auto\service\auto-service-annotations\1.0\auto-service-annotations-1.0.jar, C:\Users\User\.m2\repository\com\google\auto\service\auto-service\1.0\auto-service-1.0.jar, C:\Users\User\.m2\repository\com\google\auto\auto-common\1.0\auto-common-1.0.jar, C:\Users\User\.m2\repository\com\google\guava\guava\31.0.1-jre\guava-31.0.1-jre.jar, C:\Users\User\.m2\repository\com\google\guava\failureaccess\1.0.1\failureaccess-1.0.1.jar, C:\Users\User\.m2\repository\com\google\guava\listenablefuture\9999.0-empty-to-avoid-conflict-with-guava\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar, C:\Users\User\.m2\repository\com\google\code\findbugs\jsr305\3.0.2\jsr305-3.0.2.jar, C:\Users\User\.m2\repository\org\checkerframework\checker-qual\3.12.0\checker-qual-3.12.0.jar, C:\Users\User\.m2\repository\com\google\errorprone\error_prone_annotations\2.7.1\error_prone_annotations-2.7.1.jar, C:\Users\User\.m2\repository\com\google\j2objc\j2objc-annotations\1.3\j2objc-annotations-1.3.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-chromium-driver\4.0.0\selenium-chromium-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-json\4.0.0\selenium-json-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-devtools-v85\4.0.0\selenium-devtools-v85-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-devtools-v93\4.0.0\selenium-devtools-v93-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-devtools-v94\4.0.0\selenium-devtools-v94-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-devtools-v95\4.0.0\selenium-devtools-v95-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-edge-driver\4.0.0\selenium-edge-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-firefox-driver\4.0.0\selenium-firefox-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-firefox-xpi-driver\4.0.0\selenium-firefox-xpi-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-ie-driver\4.0.0\selenium-ie-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-opera-driver\4.0.0\selenium-opera-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-remote-driver\4.0.0\selenium-remote-driver-4.0.0.jar, C:\Users\User\.m2\repository\com\beust\jcommander\1.81\jcommander-1.81.jar, C:\Users\User\.m2\repository\io\netty\netty-buffer\4.1.68.Final\netty-buffer-4.1.68.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-codec-http\4.1.67.Final\netty-codec-http-4.1.67.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-codec\4.1.67.Final\netty-codec-4.1.67.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-handler\4.1.67.Final\netty-handler-4.1.67.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-common\4.1.68.Final\netty-common-4.1.68.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-transport-native-epoll\4.1.67.Final\netty-transport-native-epoll-4.1.67.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-transport-native-kqueue\4.1.67.Final\netty-transport-native-kqueue-4.1.67.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-transport-native-unix-common\4.1.67.Final\netty-transport-native-unix-common-4.1.67.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-transport\4.1.68.Final\netty-transport-4.1.68.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-resolver\4.1.68.Final\netty-resolver-4.1.68.Final.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-api\1.6.0\opentelemetry-api-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-context\1.6.0\opentelemetry-context-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-exporter-logging\1.6.0\opentelemetry-exporter-logging-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-sdk-metrics\1.6.0-alpha\opentelemetry-sdk-metrics-1.6.0-alpha.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-sdk-common\1.6.0\opentelemetry-sdk-common-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-sdk-extension-autoconfigure-spi\1.6.0\opentelemetry-sdk-extension-autoconfigure-spi-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-sdk-extension-autoconfigure\1.6.0-alpha\opentelemetry-sdk-extension-autoconfigure-1.6.0-alpha.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-sdk-trace\1.6.0\opentelemetry-sdk-trace-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-api-metrics\1.6.0-alpha\opentelemetry-api-metrics-1.6.0-alpha.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-sdk\1.6.0\opentelemetry-sdk-1.6.0.jar, C:\Users\User\.m2\repository\io\opentelemetry\opentelemetry-semconv\1.6.0-alpha\opentelemetry-semconv-1.6.0-alpha.jar, C:\Users\User\.m2\repository\io\ous\jtoml\2.0.0\jtoml-2.0.0.jar, C:\Users\User\.m2\repository\org\apache\commons\commons-exec\1.3\commons-exec-1.3.jar, C:\Users\User\.m2\repository\org\asynchttpclient\async-http-client\2.12.3\async-http-client-2.12.3.jar, C:\Users\User\.m2\repository\org\asynchttpclient\async-http-client-netty-utils\2.12.3\async-http-client-netty-utils-2.12.3.jar, C:\Users\User\.m2\repository\io\netty\netty-codec-socks\4.1.60.Final\netty-codec-socks-4.1.60.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-handler-proxy\4.1.60.Final\netty-handler-proxy-4.1.60.Final.jar, C:\Users\User\.m2\repository\io\netty\netty-transport-native-epoll\4.1.60.Final\netty-transport-native-epoll-4.1.60.Final-linux-x86_64.jar, C:\Users\User\.m2\repository\io\netty\netty-transport-native-kqueue\4.1.60.Final\netty-transport-native-kqueue-4.1.60.Final-osx-x86_64.jar, C:\Users\User\.m2\repository\org\reactivestreams\reactive-streams\1.0.3\reactive-streams-1.0.3.jar, C:\Users\User\.m2\repository\com\typesafe\netty\netty-reactive-streams\2.0.4\netty-reactive-streams-2.0.4.jar, C:\Users\User\.m2\repository\com\sun\activation\jakarta.activation\1.2.2\jakarta.activation-1.2.2.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-http\4.0.0\selenium-http-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-safari-driver\4.0.0\selenium-safari-driver-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\selenium-support\4.0.0\selenium-support-4.0.0.jar, C:\Users\User\.m2\repository\org\seleniumhq\selenium\htmlunit-driver\3.55.0\htmlunit-driver-3.55.0.jar, C:\Users\User\.m2\repository\net\sourceforge\htmlunit\htmlunit\2.55.0\htmlunit-2.55.0.jar, C:\Users\User\.m2\repository\xalan\xalan\2.7.2\xalan-2.7.2.jar, C:\Users\User\.m2\repository\xalan\serializer\2.7.2\serializer-2.7.2.jar, C:\Users\User\.m2\repository\org\apache\commons\commons-text\1.9\commons-text-1.9.jar, C:\Users\User\.m2\repository\org\apache\httpcomponents\httpmime\4.5.13\httpmime-4.5.13.jar, C:\Users\User\.m2\repository\net\sourceforge\htmlunit\htmlunit-core-js\2.55.0\htmlunit-core-js-2.55.0.jar, C:\Users\User\.m2\repository\net\sourceforge\htmlunit\neko-htmlunit\2.55.0\neko-htmlunit-2.55.0.jar, C:\Users\User\.m2\repository\xerces\xercesImpl\2.12.1\xercesImpl-2.12.1.jar, C:\Users\User\.m2\repository\xml-apis\xml-apis\1.4.01\xml-apis-1.4.01.jar, C:\Users\User\.m2\repository\net\sourceforge\htmlunit\htmlunit-cssparser\1.10.0\htmlunit-cssparser-1.10.0.jar, C:\Users\User\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar, C:\Users\User\.m2\repository\commons-net\commons-net\3.8.0\commons-net-3.8.0.jar, C:\Users\User\.m2\repository\org\brotli\dec\0.1.2\dec-0.1.2.jar, C:\Users\User\.m2\repository\com\shapesecurity\salvation2\3.0.0\salvation2-3.0.0.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\websocket\websocket-client\9.4.44.v20210927\websocket-client-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\jetty-client\9.4.44.v20210927\jetty-client-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\jetty-http\9.4.44.v20210927\jetty-http-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\jetty-util\9.4.44.v20210927\jetty-util-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\jetty-io\9.4.44.v20210927\jetty-io-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\websocket\websocket-common\9.4.44.v20210927\websocket-common-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\eclipse\jetty\websocket\websocket-api\9.4.44.v20210927\websocket-api-9.4.44.v20210927.jar, C:\Users\User\.m2\repository\org\apache\maven\plugins\maven-dependency-plugin\3.2.0\maven-dependency-plugin-3.2.0.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-artifact\3.1.1\maven-artifact-3.1.1.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-plugin-api\3.1.1\maven-plugin-api-3.1.1.jar, C:\Users\User\.m2\repository\org\eclipse\sisu\org.eclipse.sisu.plexus\0.0.0.M5\org.eclipse.sisu.plexus-0.0.0.M5.jar, C:\Users\User\.m2\repository\javax\enterprise\cdi-api\1.0\cdi-api-1.0.jar, C:\Users\User\.m2\repository\javax\annotation\jsr250-api\1.0\jsr250-api-1.0.jar, C:\Users\User\.m2\repository\javax\inject\javax.inject\1\javax.inject-1.jar, C:\Users\User\.m2\repository\org\sonatype\sisu\sisu-guice\3.1.0\sisu-guice-3.1.0-no_aop.jar, C:\Users\User\.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar, C:\Users\User\.m2\repository\org\eclipse\sisu\org.eclipse.sisu.inject\0.0.0.M5\org.eclipse.sisu.inject-0.0.0.M5.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-model\3.1.1\maven-model-3.1.1.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-core\3.1.1\maven-core-3.1.1.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-settings-builder\3.1.1\maven-settings-builder-3.1.1.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-model-builder\3.1.1\maven-model-builder-3.1.1.jar, C:\Users\User\.m2\repository\org\eclipse\aether\aether-impl\0.9.0.M2\aether-impl-0.9.0.M2.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-interpolation\1.19\plexus-interpolation-1.19.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-classworlds\2.5.1\plexus-classworlds-2.5.1.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-component-annotations\1.5.5\plexus-component-annotations-1.5.5.jar, C:\Users\User\.m2\repository\org\sonatype\plexus\plexus-sec-dispatcher\1.3\plexus-sec-dispatcher-1.3.jar, C:\Users\User\.m2\repository\org\sonatype\plexus\plexus-cipher\1.4\plexus-cipher-1.4.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-repository-metadata\3.1.1\maven-repository-metadata-3.1.1.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-settings\3.1.1\maven-settings-3.1.1.jar, C:\Users\User\.m2\repository\org\apache\maven\maven-aether-provider\3.1.1\maven-aether-provider-3.1.1.jar, C:\Users\User\.m2\repository\org\eclipse\aether\aether-spi\0.9.0.M2\aether-spi-0.9.0.M2.jar, C:\Users\User\.m2\repository\org\apache\maven\reporting\maven-reporting-api\3.0\maven-reporting-api-3.0.jar, C:\Users\User\.m2\repository\org\apache\maven\reporting\maven-reporting-impl\3.0.0\maven-reporting-impl-3.0.0.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-decoration-model\1.7.4\doxia-decoration-model-1.7.4.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-sink-api\1.9.1\doxia-sink-api-1.9.1.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-logging-api\1.9.1\doxia-logging-api-1.9.1.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-core\1.9.1\doxia-core-1.9.1.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-container-default\1.7.1\plexus-container-default-1.7.1.jar, C:\Users\User\.m2\repository\org\apache\xbean\xbean-reflect\3.7\xbean-reflect-3.7.jar, C:\Users\User\.m2\repository\com\google\collections\google-collections\1.0\google-collections-1.0.jar, C:\Users\User\.m2\repository\org\apache\httpcomponents\httpclient\4.5.8\httpclient-4.5.8.jar, C:\Users\User\.m2\repository\commons-codec\commons-codec\1.11\commons-codec-1.11.jar, C:\Users\User\.m2\repository\org\apache\httpcomponents\httpcore\4.4.11\httpcore-4.4.11.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-site-renderer\1.9.2\doxia-site-renderer-1.9.2.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-skin-model\1.9.2\doxia-skin-model-1.9.2.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-module-xhtml\1.9.1\doxia-module-xhtml-1.9.1.jar, C:\Users\User\.m2\repository\org\apache\maven\doxia\doxia-module-xhtml5\1.9.1\doxia-module-xhtml5-1.9.1.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-i18n\1.0-beta-10\plexus-i18n-1.0-beta-10.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-velocity\1.2\plexus-velocity-1.2.jar, C:\Users\User\.m2\repository\org\apache\velocity\velocity\1.7\velocity-1.7.jar, C:\Users\User\.m2\repository\commons-lang\commons-lang\2.4\commons-lang-2.4.jar, C:\Users\User\.m2\repository\org\apache\velocity\velocity-tools\2.0\velocity-tools-2.0.jar, C:\Users\User\.m2\repository\commons-beanutils\commons-beanutils\1.7.0\commons-beanutils-1.7.0.jar, C:\Users\User\.m2\repository\commons-digester\commons-digester\1.8\commons-digester-1.8.jar, C:\Users\User\.m2\repository\commons-chain\commons-chain\1.1\commons-chain-1.1.jar, C:\Users\User\.m2\repository\dom4j\dom4j\1.1\dom4j-1.1.jar, C:\Users\User\.m2\repository\oro\oro\2.0.8\oro-2.0.8.jar, C:\Users\User\.m2\repository\commons-collections\commons-collections\3.2.2\commons-collections-3.2.2.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-archiver\4.2.2\plexus-archiver-4.2.2.jar, C:\Users\User\.m2\repository\org\apache\commons\commons-compress\1.20\commons-compress-1.20.jar, C:\Users\User\.m2\repository\org\iq80\snappy\snappy\0.4\snappy-0.4.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-utils\3.3.0\plexus-utils-3.3.0.jar, C:\Users\User\.m2\repository\org\codehaus\plexus\plexus-io\3.2.0\plexus-io-3.2.0.jar, C:\Users\User\.m2\repository\org\apache\maven\shared\maven-dependency-analyzer\1.11.3\maven-dependency-analyzer-1.11.3.jar, C:\Users\User\.m2\repository\org\apache\maven\shared\maven-dependency-tree\3.0.1\maven-dependency-tree-3.0.1.jar, C:\Users\User\.m2\repository\org\apache\maven\shared\maven-common-artifact-filters\3.2.0\maven-common-artifact-filters-3.2.0.jar, C:\Users\User\.m2\repository\org\apache\maven\shared\maven-artifact-transfer\0.13.1\maven-artifact-transfer-0.13.1.jar, C:\Users\User\.m2\repository\org\apache\maven\shared\maven-shared-utils\3.3.4\maven-shared-utils-3.3.4.jar, C:\Users\User\.m2\repository\org\apache\commons\commons-collections4\4.2\commons-collections4-4.2.jar, C:\Users\User\.m2\repository\classworlds\classworlds\1.1\classworlds-1.1.jar, C:\Users\User\.m2\repository\org\eclipse\aether\aether-api\0.9.0.M2\aether-api-0.9.0.M2.jar, C:\Users\User\.m2\repository\org\eclipse\aether\aether-util\0.9.0.M2\aether-util-0.9.0.M2.jar, C:\Users\User\.m2\repository\org\hamcrest\hamcrest\2.2\hamcrest-2.2.jar], mutators=[CONDITIONALS_BOUNDARY, INCREMENTS, INVERT_NEGS, MATH, NEGATE_CONDITIONALS, VOID_METHOD_CALLS, EMPTY_RETURNS, FALSE_RETURNS, TRUE_RETURNS, NULL_RETURNS, PRIMITIVE_RETURNS], features=[], dependencyAnalysisMaxDistance=-1, jvmArgs=[-Djava.awt.headless=true], numberOfThreads=1, timeoutFactor=1.25, timeoutConstant=4000, targetTests=[^delft\.MathArraysPBTest$], loggingClasses=[], verbose=true, failWhenNoMutations=true, outputs=[HTML], groupConfig=TestGroupConfig [excludedGroups=[], includedGroups=[]], fullMutationMatrix=false, mutationUnitSize=0, shouldCreateTimestampedReports=true, detectInlinedCode=false, exportLineCoverage=false, mutationThreshold=0, testStrengthThreshold=0, coverageThreshold=0, mutationEngine=gregor, javaExecutable=null, includeLaunchClasspath=false, properties={}, maxSurvivors=-1, excludedRunners=[], includedTestMethods=[], testPlugin=junit, useClasspathJar=false, skipFailingTests=false]22:45:55 PIT >> FINE : System class path is C:\Users\Paul\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\212.5284.40\plugins\maven\lib\maven3\boot\plexus-classworlds-2.6.0.jar;C:\Users\Paul\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\212.5284.40\plugins\maven\lib\maven3\boot\plexus-classworlds.license
22:45:55 PIT >> FINE : Maximum available memory is 1024 mb
22:45:55 PIT >> FINE : Incremental analysis set 6 mutations to a status of NOT_STARTED
22:45:55 PIT >> INFO : Incremental analysis reduced number of mutations by 0
22:45:55 PIT >> INFO : Created  1 mutation test units in pre scan
22:45:55 PIT >> FINE : MINION : Installing PIT agent
22:45:55 PIT >> INFO : Sending 1 test classes to minion
22:45:55 PIT >> INFO : Sent tests to minion
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> FINE : Expecting 1 tests classes from parent
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> FINE : Tests classes received
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> INFO : Checking environment
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> INFO : Found  0 tests
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> INFO : Dependency analysis reduced number of potential tests by 0
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> INFO : 0 tests received
22:45:55 PIT >> INFO : MINION : 22:45:55 PIT >> FINE : Running 0 units
22:45:55 PIT >> FINE : Coverage generator Minion exited ok
22:45:55 PIT >> INFO : Calculated coverage in 0 seconds.
22:45:55 PIT >> FINE : Used memory after coverage calculation 45 mb
22:45:55 PIT >> FINE : Free Memory after coverage calculation 56 mb
22:45:56 PIT >> FINE : Incremental analysis set 6 mutations to a status of NOT_STARTED
22:45:56 PIT >> INFO : Incremental analysis reduced number of mutations by 0
22:45:56 PIT >> INFO : Created  1 mutation test units
22:45:56 PIT >> FINE : Used memory before analysis start 63 mb
22:45:56 PIT >> FINE : Free Memory before analysis start 38 mb
22:45:56 PIT >> FINE : Running 1 units
22:45:56 PIT >> INFO : Completed in 1 seconds
================================================================================
- Mutators
================================================================================
> org.pitest.mutationtest.engine.gregor.mutators.ConditionalsBoundaryMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.IncrementsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NullReturnValsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.MathMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
================================================================================
- Timings
================================================================================
> pre-scan for mutations : < 1 second
> scan classpath : < 1 second
> coverage and dependency analysis : < 1 second
> build mutation tests : < 1 second
> run mutation analysis : < 1 second
--------------------------------------------------------------------------------
> Total  : < 1 second
--------------------------------------------------------------------------------
================================================================================
- Statistics
================================================================================
>> Line Coverage: 0/12 (0%)
>> Generated 6 mutations Killed 0 (0%)
>> Mutations with no coverage 6. Test strength 0%
>> Ran 0 tests (0 tests per mutation)

Unfortunately we have a lot of libraries, but a quick CTRL+F will show you that Jqwik, JUnit API/Engine and the plugin are present.

This is the test class that is being run:

class MathArraysPBTest {

    @Property
    // an array of size 100 with doubles between [1,20] will definitely have repeated numbers
    // if you use only @ForAll, you are likely to never have repeated elements in the array, and thus,
    // never exercising the full property!
    void unique( 
            @ForAll
            @Size(value = 100)
            List<@DoubleRange(min = 1, max = 20) Double> numbers) {

        double[] doubles = convertListToArray(numbers);
        double[] result = MathArrays.unique(doubles);

        assertThat(result)
                .contains(doubles) // contains all the elements
                .doesNotHaveDuplicates() // no duplicates
                .isSortedAccordingTo(Comparator.reverseOrder()); // in descending order
    }

    /** Use this method to convert a list of integers to an array */
    private double[] convertListToArray(List<Double> numbers) {
        double[] array = numbers.stream().mapToDouble(x -> x).toArray();
        return array;
    }
}
Arraying commented 2 years ago

I found a fix. By default jqwik includes its engine as a runtime only dependency. It did not end up in PIT's ReportOption's classPathElements for us. Ensuring that jqwik-engine-X.X.X.jar was present in there solved the problem. For us, this meant explicitly setting the jqwik-engine dependency's <scope> to compile. YMMV.