pitest / pitclipse

Mutation testing for Java in Eclipse IDE. Based on PIT (Pitest).
https://pitest.org
Apache License 2.0
59 stars 17 forks source link

Flaky UI tests #204

Closed LorenzoBettini closed 2 years ago

LorenzoBettini commented 2 years ago

Some UI tests involving the "PIT Mutations" view are flaky, especially in Windows and macOS. The toolbar buttons cannot be found, but that is due to the view itself not being active.

In particular, this condition does not seem to make sense:

class ViewOpenedCondition extends DefaultCondition {

    private SWTWorkbenchBot bot;
    private String viewTitle;

    public ViewOpenedCondition(SWTWorkbenchBot bot, String viewTitle) {
        this.bot = bot;
        this.viewTitle = viewTitle;
    }

    @Override
    public boolean test() throws Exception {
        return bot.viewByTitle(viewTitle) != null;
    }

since the return value of viewByTitle cannot be null. Instead, if we use as condition isActive on the returned view, we get the failing condition.

The screenshots upon failures are of this shape (macOS, Windows and Linux, respectively):

image

image

image

meaning that the "PIT Mutations" view is not active at all.

Investigating in #203

LorenzoBettini commented 2 years ago

Closed by #203