authorjapps / zerocode

A community-developed, free, opensource, automated testing framework for microservices APIs, Kafka(Data Streams) and Load testing. Zerocode Open Source enables you to create, change and maintain your automated test scenarios via simple JSON or YAML files. Visit documentation below:
https://zerocode-tdd.tddfy.com
Apache License 2.0
909 stars 397 forks source link

Fixed flaky test-case: ZeroCodeAssertionsProcessorImplTest#testDate.testDateAfterBefore_fail_both #691

Open hermya opened 1 month ago

hermya commented 1 month ago

Fixed flaky test-case

Fixes Issue

PR Branch fork

Motivation and Context

ZeroCodeAssertionsProcessorImplTest#testDate.testDateAfterBefore_fail_both is a flaky test, fixed in this PR.

POINT OF FAILURE ->

In ZeroCodeAssertionsProcessorImpl.createJsonAsserters, ObjectMapper.mapTree creates a JsonNode object whose key-value pairs are not necessarily ordered, thus the elements in List<JsonAsserter> are not ordered, causing further failures. Fixed using NonDex pugin

For particular test: Add the following lines to your pom.xml

  <plugin> <!-- for flaky testing -->
      <groupId>edu.illinois</groupId>
      <artifactId>nondex-maven-plugin</artifactId>
      <version>2.1.7</version>
  </plugin>

and run

mvn -pl ./core nondex:nondex -Dtest=org.jsmart.zerocode.core.engine.preprocessor.ZeroCodeAssertionsProcessorImplTest#testDateAfterBefore_fail_both -DnondexRuns=10 -DnondexMode=ONE

OR

mvn -pl ./core edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=org.jsmart.zerocode.core.engine.preprocessor.ZeroCodeAssertionsProcessorImplTest#testDateAfterBefore_fail_both -DnondexRuns=10 -DnondexMode=ONE

For more information : https://github.com/TestingResearchIllinois/NonDex

Checklist: