dianping / cat

CAT 作为服务端项目基础组件,提供了 Java, C/C++, Node.js, Python, Go 等多语言客户端,已经在美团点评的基础架构中间件框架(MVC框架,RPC框架,数据库框架,缓存框架等,消息队列,配置系统等)深度集成,为美团点评各业务线提供系统丰富的性能指标、健康状况、实时告警等。
Apache License 2.0
18.63k stars 5.42k forks source link

fix flakiness in test com.dianping.cat.message.context.MessageIdFactoryTest#testDefaultDomainInParallel #2317

Open hofi1 opened 11 months ago

hofi1 commented 11 months ago

Problem:

This test is flaky due to the fact that the time which is provided (to shut the pool down) can be sufficient or not. It can fail to shut the pool down in the provided time (what results in a failed test) If the code remains the way it is, it results in a flaky test which sometimes passes and might fail other times (non-deterministic behavior).

Solution:

I added the condition if the shutdown is finished or if it failed. Furthermore, the duration for the shutdown got increased to make sure, that it is possible to finish the process without running in a timeout. If the shutdown reaches the timeout, the test fails as intended. If the shutdown is finished successfully, the same conditions (asserts) apply as before.

Result:

The test is deterministic and not flaky. This improves the quality of the test and reduces the time to search for the bug during future development.