search

invertase / react-native-firebase

🔥 A well-tested feature-rich modular Firebase implementation for React Native. Supports both iOS & Android platforms for all Firebase services.
https://rnfirebase.io
Other
11.62k stars 2.2k forks source link

CI is not executing all the tests all the time #7906

Open mikehardy opened 1 month ago

mikehardy commented 1 month ago 
          CI is not executing all the tests all the time as noted on slack, that's a pretty serious problem as false-positives on the testing are dangerous. I noticed it on this PR since the codecov bot annotated lines as uncovered when a test clearly covered them.

Local testing is passing but we need to find the root cause of CI not executing all tests


        ✔ it should create js apps for natively initialized apps
[🟨] Jet client disconnected - normal closure (code = 1000).
[🟩] Jet client connected from "127.0.0.1:40293".

  18 passing (4s)

Originally posted by @mikehardy in https://github.com/invertase/react-native-firebase/pull/7897#pullrequestreview-2177795050

mikehardy commented 1 month ago

In other repos, I've done an assertion that a non-zero number of tests executed, but that's not a great solution here as it's a semi-random non-zero already

We could assert on a reasonable minimum number - for instance right now we expect 2305 passing / 153 skipped in local android runs for instance. iOS is likely similar? We could assert 2200+ ran or something

Alternatively we could determine the root cause and look for that and assert that the root cause did not happen at any time