search

FroMage / redpipe

Redpipe Web Framework
Apache License 2.0
70 stars 10 forks source link

Fix flaky test based on Nondex #56

Open SpiceOwlf opened 2 years ago

SpiceOwlf commented 2 years ago

Hi, I used Nondex(https://github.com/TestingResearchIllinois/NonDex) on the project and found [checkTemplateNegociationDefault] does not have a uniform behavior. So I made some changes to the test to make it pass all possible situations. Please let me know if you want to discuss this change more. Thanks.

SpiceOwlf commented 2 years ago

Updates: The root cause of the failure is that (1) the templates are read from thedisk using listFiles() in https://github.com/FroMage/redpipe/blob/0aff891d6befdf0dcc2bdbfda22262cdf82ac66f/redpipe-engine/src/main/java/net/redpipe/engine/template/AbstractTemplate.java#L88 and (2) the files are put in a HashMap of variants in https://github.com/FroMage/redpipe/blob/0aff891d6befdf0dcc2bdbfda22262cdf82ac66f/redpipe-engine/src/main/java/net/redpipe/engine/template/AbstractTemplate.java#L94 Neither listFiles() nor HashMap guarantee the order of the elements, so the order of the variants in nondeterministic. When the default variant is chosen as the first from the map it can end up being one of the two variants, while the assertion expects it to be only one.