search

FlowCrypt / flowcrypt-ios

FlowCrypt iOS App
https://flowcrypt.com
Other
33 stars 10 forks source link

Unstable SemaphoreCI tests #2638

Closed ioanmo226 closed 4 days ago

ioanmo226 commented 2 weeks ago

Tests are consistently failing in SemaphoreCI, making it difficult to achieve a passing result. As a result, there are many PRs where tests pass locally but fail in SemaphoreCI.

We need to stabilize and fix these unstable tests in SemaphoreCI.

ioanmo226 commented 2 weeks ago

Hmm... SemaphoreCI tests have become incredibly unstable after we took a break from the iOS app. It's almost impossible to get a passing result for any PRs now. Trying to fix unstable tests but not going well. @sosnovsky, any idea why SemaphoreCI tests are so unstable? They seemed stable before.

sosnovsky commented 2 weeks ago

Difficult to say what's exact reason, but based on other CI feedback - it can happen because iOS simulator needs powerful software, while CI machines has less resources than our own macbooks, so locally tests pass well while on CI they randomly fail. I thought using arm64 machines will fix it, as they give a big performance improvement, but unfortunately they currently aren't able to run iOS simulator. For our current configuration we can try to switch to less resource-consuming iOS simulator - iPhone SE (3rd generation) instead of iPhone 15.

ioanmo226 commented 2 weeks ago

It sounds good