xtf-cz / xtf

MIT License
12 stars 55 forks source link

SkipForCondition continue in test if skip condition cant be evaluated #591

Open mchoma opened 1 month ago

mchoma commented 1 month ago

When for skip condition is used product id, subid which are not enabled in testsuite run. SkipCondition evaluation now fail

Caused by: cz.xtf.core.image.UnknownImageException: Unable to get image using wf or 
    at cz.xtf.core.image.Image.resolve(Image.java:45)
    at cz.xtf.junit5.extensions.SkipForCondition.resolve(SkipForCondition.java:49)
    at cz.xtf.junit5.extensions.SkipForCondition.evaluateExecutionCondition(SkipForCondition.java:30)
    at org.junit.jupiter.engine.execution.ConditionEvaluator.evaluate(ConditionEvaluator.java:64)

This issue is about to continue in test in such case.

Why are we hitting this now? Now as we are going to have 2 images for WF and EAP I propose to split xtf configuration to separate xtf ids xtf.eap and xtf.wf so we can better share between those images common configuration. One image or another is used to run testsuite. It does not have sense to have set both.

So when I do that and run with xtf.eap profile @SkipFor(image="wf" is failing because no such image is configured. This issue is about enabling test instead of failing in this case.