search

getsentry / sentry-react-native

Official Sentry SDK for React Native
https://sentry.io
MIT License
1.58k stars 338 forks source link

fix(e2e): Add longer timeout for test app start #4297

Closed krystofwoldrich closed 2 days ago

krystofwoldrich commented 2 days ago

This PR adds longer timeout for test ready text to appear.

Especially RN 0.65 on iOS frequently timesout on the current 2 mins timeout.

skip-changelog

github-actions[bot] commented 2 days ago

Android (legacy) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 444.98 ms 433.45 ms -11.53 ms
Size 17.74 MiB 20.08 MiB 2.35 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
d197b5c9ca10ac3f729d1b3ef805328665aa0895+dirty 338.94 ms 354.87 ms 15.93 ms
0db0c729a4ac7c205a44cb4702fa7ba5b0bc03c0 372.12 ms 386.00 ms 13.88 ms
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 430.68 ms 456.13 ms 25.44 ms
5571a202e0c1fa9393e62c66ebd908444fee9c6e 410.55 ms 441.06 ms 30.51 ms
80b2ce3d0ebe86bf1196944a2036912f6670295c 385.02 ms 387.36 ms 2.34 ms
a5d86e1af44c3343f2be2a5463d284aaba77f3de 465.06 ms 457.83 ms -7.23 ms
7bc4d758f2bdfa0bbb51d2a9119418c0f500e893 488.76 ms 473.28 ms -15.48 ms
34aba0802bf802288f91fada55c0b133252fe67f 328.10 ms 342.84 ms 14.74 ms
52c056204eb6a8e4fe6e138e9307f2838242b4a6 453.04 ms 434.71 ms -18.33 ms
148f9248007774c638943e03dc044cf71dec0c3d 492.65 ms 500.28 ms 7.63 ms

App size

Revision Plain With Sentry Diff
d197b5c9ca10ac3f729d1b3ef805328665aa0895+dirty 17.73 MiB 20.04 MiB 2.31 MiB
0db0c729a4ac7c205a44cb4702fa7ba5b0bc03c0 17.73 MiB 19.75 MiB 2.02 MiB
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 17.73 MiB 19.75 MiB 2.01 MiB
5571a202e0c1fa9393e62c66ebd908444fee9c6e 17.73 MiB 19.93 MiB 2.19 MiB
80b2ce3d0ebe86bf1196944a2036912f6670295c 17.73 MiB 19.75 MiB 2.02 MiB
a5d86e1af44c3343f2be2a5463d284aaba77f3de 17.74 MiB 20.08 MiB 2.34 MiB
7bc4d758f2bdfa0bbb51d2a9119418c0f500e893 17.74 MiB 20.08 MiB 2.34 MiB
34aba0802bf802288f91fada55c0b133252fe67f 17.73 MiB 19.80 MiB 2.07 MiB
52c056204eb6a8e4fe6e138e9307f2838242b4a6 17.73 MiB 20.11 MiB 2.38 MiB
148f9248007774c638943e03dc044cf71dec0c3d 17.73 MiB 19.94 MiB 2.21 MiB
github-actions[bot] commented 2 days ago

Android (new) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 381.26 ms 421.45 ms 40.19 ms
Size 7.15 MiB 8.35 MiB 1.21 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
d197b5c9ca10ac3f729d1b3ef805328665aa0895+dirty 258.75 ms 313.61 ms 54.86 ms
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 371.40 ms 377.70 ms 6.31 ms
3ffcddd6248046202afca78f6b9af8e3f591202c+dirty 244.82 ms 295.47 ms 50.65 ms
ad6c2993f6b77abbcf4094d17dd1fa9f7742ae99+dirty 336.47 ms 362.89 ms 26.42 ms
8ae23a70cf23a4f7a3c54784f496e7af813b7a48+dirty 398.10 ms 411.48 ms 13.38 ms
a5d86e1af44c3343f2be2a5463d284aaba77f3de+dirty 380.86 ms 466.42 ms 85.56 ms
d2c32bbdb8e91690865169f98c6ef7e2ed672df2+dirty 445.45 ms 497.85 ms 52.41 ms
0d3e6775185b3030074b2bca3cb796e5d265910b+dirty 384.24 ms 431.45 ms 47.21 ms
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 373.98 ms 394.08 ms 20.10 ms
27ef4ee6faa6d5229b16fca7b728ca6b689ab916+dirty 296.71 ms 351.00 ms 54.29 ms

App size

Revision Plain With Sentry Diff
d197b5c9ca10ac3f729d1b3ef805328665aa0895+dirty 7.15 MiB 8.09 MiB 962.72 KiB
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 7.15 MiB 8.03 MiB 901.79 KiB
3ffcddd6248046202afca78f6b9af8e3f591202c+dirty 7.15 MiB 8.04 MiB 912.17 KiB
ad6c2993f6b77abbcf4094d17dd1fa9f7742ae99+dirty 7.15 MiB 8.04 MiB 912.17 KiB
8ae23a70cf23a4f7a3c54784f496e7af813b7a48+dirty 7.15 MiB 8.35 MiB 1.20 MiB
a5d86e1af44c3343f2be2a5463d284aaba77f3de+dirty 7.15 MiB 8.35 MiB 1.20 MiB
d2c32bbdb8e91690865169f98c6ef7e2ed672df2+dirty 7.15 MiB 8.35 MiB 1.20 MiB
0d3e6775185b3030074b2bca3cb796e5d265910b+dirty 7.15 MiB 8.35 MiB 1.20 MiB
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 7.15 MiB 8.04 MiB 910.85 KiB
27ef4ee6faa6d5229b16fca7b728ca6b689ab916+dirty 7.15 MiB 8.08 MiB 959.49 KiB
github-actions[bot] commented 2 days ago

iOS (new) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 1237.94 ms 1234.17 ms -3.77 ms
Size 2.92 MiB 3.66 MiB 758.59 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
c639edf12520d1afee79e1ff2ba18df03ce70ec3+dirty 1223.63 ms 1227.98 ms 4.35 ms
86d6d2c81bc42209de70f7f3b97a1fbbde04025f+dirty 1291.62 ms 1296.80 ms 5.18 ms
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 1220.87 ms 1221.47 ms 0.60 ms
4a6664fa7140687cfa079f2d738b59b18e4ac7b7+dirty 1218.77 ms 1221.07 ms 2.30 ms
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 1288.10 ms 1289.54 ms 1.44 ms
76d1bafa51ac0d20b127bc97b0f45201263636c3+dirty 1245.00 ms 1257.76 ms 12.76 ms
9433f356583b99b5c970842ca1f926315bf23ba9+dirty 1232.24 ms 1232.74 ms 0.50 ms
0d3e6775185b3030074b2bca3cb796e5d265910b+dirty 1239.02 ms 1241.22 ms 2.20 ms
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 1280.06 ms 1285.26 ms 5.20 ms
abb7058e207628f7d3e5ebee6c410c2cf6c361f4+dirty 1260.28 ms 1266.56 ms 6.28 ms

App size

Revision Plain With Sentry Diff
c639edf12520d1afee79e1ff2ba18df03ce70ec3+dirty 2.92 MiB 3.64 MiB 742.55 KiB
86d6d2c81bc42209de70f7f3b97a1fbbde04025f+dirty 2.92 MiB 3.37 MiB 464.31 KiB
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 2.92 MiB 3.43 MiB 529.76 KiB
4a6664fa7140687cfa079f2d738b59b18e4ac7b7+dirty 2.92 MiB 3.60 MiB 702.09 KiB
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 2.92 MiB 3.40 MiB 488.06 KiB
76d1bafa51ac0d20b127bc97b0f45201263636c3+dirty 2.92 MiB 3.38 MiB 475.74 KiB
9433f356583b99b5c970842ca1f926315bf23ba9+dirty 2.92 MiB 3.41 MiB 503.55 KiB
0d3e6775185b3030074b2bca3cb796e5d265910b+dirty 2.92 MiB 3.66 MiB 758.42 KiB
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 2.92 MiB 3.44 MiB 533.26 KiB
abb7058e207628f7d3e5ebee6c410c2cf6c361f4+dirty 2.92 MiB 3.43 MiB 524.53 KiB
github-actions[bot] commented 2 days ago

iOS (legacy) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 1222.34 ms 1241.43 ms 19.09 ms
Size 2.36 MiB 3.10 MiB 753.36 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
c639edf12520d1afee79e1ff2ba18df03ce70ec3+dirty 1236.18 ms 1235.04 ms -1.14 ms
86d6d2c81bc42209de70f7f3b97a1fbbde04025f+dirty 1267.55 ms 1286.21 ms 18.66 ms
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 1225.08 ms 1230.26 ms 5.17 ms
4a6664fa7140687cfa079f2d738b59b18e4ac7b7+dirty 1209.49 ms 1208.63 ms -0.86 ms
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 1252.38 ms 1275.04 ms 22.66 ms
76d1bafa51ac0d20b127bc97b0f45201263636c3+dirty 1244.10 ms 1268.52 ms 24.42 ms
9433f356583b99b5c970842ca1f926315bf23ba9+dirty 1246.94 ms 1271.45 ms 24.52 ms
0d3e6775185b3030074b2bca3cb796e5d265910b+dirty 1214.39 ms 1225.70 ms 11.31 ms
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 1256.10 ms 1259.08 ms 2.98 ms
abb7058e207628f7d3e5ebee6c410c2cf6c361f4+dirty 1255.42 ms 1268.86 ms 13.44 ms

App size

Revision Plain With Sentry Diff
c639edf12520d1afee79e1ff2ba18df03ce70ec3+dirty 2.36 MiB 3.08 MiB 736.63 KiB
86d6d2c81bc42209de70f7f3b97a1fbbde04025f+dirty 2.36 MiB 2.82 MiB 462.82 KiB
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 2.36 MiB 2.88 MiB 525.47 KiB
4a6664fa7140687cfa079f2d738b59b18e4ac7b7+dirty 2.36 MiB 3.04 MiB 696.39 KiB
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 2.36 MiB 2.83 MiB 481.14 KiB
76d1bafa51ac0d20b127bc97b0f45201263636c3+dirty 2.36 MiB 2.82 MiB 469.45 KiB
9433f356583b99b5c970842ca1f926315bf23ba9+dirty 2.36 MiB 2.85 MiB 499.80 KiB
0d3e6775185b3030074b2bca3cb796e5d265910b+dirty 2.36 MiB 3.10 MiB 753.12 KiB
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 2.36 MiB 2.88 MiB 530.38 KiB
abb7058e207628f7d3e5ebee6c410c2cf6c361f4+dirty 2.36 MiB 2.87 MiB 520.42 KiB