search

authorjapps / zerocode

A community-developed, free, opensource, automated testing framework for microservices APIs, Kafka(Data Streams) and Load testing. Zerocode Open Source enables you to create, change and maintain your automated test scenarios via simple JSON or YAML files. Visit documentation below:
https://zerocode-tdd.tddfy.com
Apache License 2.0
909 stars 397 forks source link

Flaky tests detected: ConsumerJsonRecordTest.should_serialize_a_record_with_headers #685

Closed hermya closed 1 month ago

hermya commented 1 month ago

The Problem:

There are some flaky-tests in the repo, which assert the output of non-deterministic outputs. They may pass in specific environment but possible fail in others. By using nondex, one can find such cases.

For example: ConsumerJsonRecordTest.should_serialize_a_record_with_headers fails intermittently

Expected: "{\"key\":123,\"value\":\"val\",\"headers\":{\"hKey\":\"hValue\",\"hKeyWithNullValue\":null}}"
     but: was "{\"key\":123,\"value\":\"val\",\"headers\":{\"hKeyWithNullValue\":null,\"hKey\":\"hValue\"}}"

Notice the string mismatch due to change in ordering of keys: {hKey, hKeyWithNullValue} <=> {hKeyWithNullValue, hKey}

About The Fix:

Answered here.

nirmalchandra commented 1 month ago

PR merge. Changes were only to Test classes, so no impact to the release versions.