kirbydesign / designsystem

Kirby Design System
https://cookbook.kirby.design
MIT License
85 stars 23 forks source link

[Housekeeping] Harden flakey test #1836

Closed RasmusKjeldgaard closed 2 years ago

RasmusKjeldgaard commented 3 years ago

Have you considered any alternatives?

Could we either avoid using actual time (some fakeAsync magic or similar?) or otherwise consider to add more milliseconds to the timeout in dropdown.popover.component.spec.ts

Are there any additional context?

image

MadsBuchmann commented 3 years ago

@RasmusKjeldgaard let's just keep this as an housekeeping issue. As long as it is documented 😄

MadsBuchmann commented 2 years ago

I have taken the liberty of changing the title of this issue in order to make it a general issue for documenting flakey tests. I think they should all be fixed in one go.

I know @troelslenda and @mictro has had trouble with the one below from libs/designsystem/src/lib/components/modal/services/action-sheet.helper.spec.ts:

it('backdrop click should dismiss action-sheet', async () => {
      let modalDidDismiss = false;
      ionModal.onDidDismiss().then((_) => (modalDidDismiss = true));
      backdrop.dispatchEvent(new MouseEvent('click'));
      await new Promise<void>((resolve) => setTimeout(resolve, 15));
      expect(modalDidDismiss).toBeTrue();
});

I haven't been able to recreate either of those on my own. It might be related to M1 Macs.

Edit: I've gotten the 'backdrop click should dismiss action-sheet' error on my own machine now. My machine is a Mac with runs a intel processor, so it is probably not related to the machines using M1's.

RasmusKjeldgaard commented 2 years ago

Just adding this workflow run to the pile of possible flakey tests. First time seeing this on the CI, for me at least, so we should investigate if something changed around this time. The failing tests are all related to the ModalHelper showModalWindow method.

RasmusKjeldgaard commented 2 years ago

This test which is also linked above, started failing more consistently during the Angular upgrade. It also happens on the CI environment now.

For now solved by slightly bumping the timeout from 15 to 20, but of course not ideal. Another testament to how flaky this way of doing it is, if small performance differences from version to version of Angular can affect this.

MadsBuchmann commented 2 years ago

Another dropdown test that keeps failing and has been excluded for now.

Should be reintroduced as part of this issue.