Intermittent test failure in file browser test

[romanofski]

Describe the bug I've had a test run which seemed to have failed with an intermittent failure, since it only happend on the nix build. The last screenshot seems to suggest that for whatever reason the list item wasn't selecting the last remaining entity to trigger the error message.

Perhaps it is again a too generic intermittent step which causes the tests to skip too quickly?

To Reproduce No idea at this point.

Expected behavior Test run without a failure

Screenshots See travis build log.txt

Additional context None

[romanofski]

Happened today again:

[romanofski]

Happened again here: https://travis-ci.org/purebred-mua/purebred/jobs/606239891?utm_medium=notification&utm_source=email

[romanofski]

I've looked into this a little bit today. The test steps seem fine and I can't find any sub strings which are too generic so the test run would skip ahead.

The only hypothesis I could come up with is perhaps the attachment list and it's length indicator in the status bar. Our test framework doesn't really let us assert that a string has been removed from the previous screen (maybe a feature we should implement?). Instead of asserting that the last file was deleted, because it's now missing, we use the item indicator in the status bar and assert with the list length, e.g. Item 1 of 2.

Now I'm wondering whether perhaps the list length is recalculated quicker (since it's asynchronous) than the redrawing of the entire UI can take place. In the screen shot the deleted list entry still looks like it's selected, even tho the entry is gone.

If that's a theory to pursue, I'm currently not sure what else to assert on which would be a stable indicator that the file is not in the list/UI.

@frasertweedale does that sound plausible as a possible cause?

[romanofski]

Happened again today: https://travis-ci.org/purebred-mua/purebred/jobs/613420288?utm_medium=notification&utm_source=github_status

[romanofski]

Had this come up again today in a test failure: https://travis-ci.org/purebred-mua/purebred/jobs/629258818?utm_medium=notification&utm_source=email

Unfortunately it seems, this issue isn't fixed yet :/

[romanofski]

[romanofski]

Looking into this closer, I wonder whether there was a slight chance that I had made the branch in which the above error appeared before I merged this fix. While I authored the fix in November, we kept it around for a while. I haven't seen the problem re-occur on master, so perhaps we keep this one open for another month or so and if it really doesn't appear again I'll close it.

[romanofski]

This hasn't resurfaced in a few weeks now making me think that I must have created my branch with the problem in it. I'll close the issue.

[romanofski]

This job https://travis-ci.org/github/purebred-mua/purebred/jobs/675540114 seems to have failed at the same point.

[romanofski]

Saw the failure again today. Attaching log.txt

[romanofski]

Happened again today. Attaching log.txt

[romanofski]

Haha look who's back: https://travis-ci.org/github/purebred-mua/purebred/jobs/716221854