Looks like after updating some of runners to XCode 15.3 booting up a simulator takes long time (actually it boots pretty fast, but after that CPU usage is close to 100% for quite big period of time). As a result tests are failing because of timeout.
I've tried many things - switching to other XCode versions, increasing timeout, changing the order of tests, changing runners, pre-booting simultator etc. and still tests very flaky in some cases.
However what I discovered is that after ~20mins and after several failures tests start to work okay. So I decided to add --retry 2 flag to overcome the problem of flaky simulator - it makes CI green again. But if a regression really introduced - it'll fail anyway just with linger time of execution π
π’ Changelog
E2E
added patch-package;
updated detox;
updated assets;
moved tests preparation in jest-setup.ts file;
run tests with retries;
use scroll instead of swipe in AwareScrollView test;
CI
increased timeout (60 -> 90);
π€ How Has This Been Tested?
Tested on CI.
π Checklist
[x] CI successfully passed
[x] I added new mocks and corresponding unit-tests if library API was changed
π Description
Make iOS tests on CI stable again.
π‘ Motivation and Context
Looks like after updating some of runners to XCode 15.3 booting up a simulator takes long time (actually it boots pretty fast, but after that CPU usage is close to 100% for quite big period of time). As a result tests are failing because of timeout.
I've tried many things - switching to other XCode versions, increasing timeout, changing the order of tests, changing runners, pre-booting simultator etc. and still tests very flaky in some cases.
However what I discovered is that after ~20mins and after several failures tests start to work okay. So I decided to add
--retry 2
flag to overcome the problem of flaky simulator - it makes CI green again. But if a regression really introduced - it'll fail anyway just with linger time of execution ππ’ Changelog
E2E
patch-package
;detox
;jest-setup.ts
file;AwareScrollView
test;CI
timeout
(60 -> 90);π€ How Has This Been Tested?
Tested on CI.
π Checklist