feat: use `addTimingsCallback` instead of `addPostFrameCallback` to determine app start end

buenaflor commented 2 weeks ago

:scroll: Description

Improve app start measurements

:bulb: Motivation and Context

The addTimingsCallback contains the accurate data for the first frame unlike addPostFramecallback which is not correct as it depends on when SentryFlutter.init is called

addTimingsCallback is always reported immediately for the first frame so it's very suitable to use here, see https://api.flutter.dev/flutter/scheduler/SchedulerBinding/addTimingsCallback.html

:green_heart: How did you test it?

Unit tests, manual tests

:pencil: Checklist

[x] I reviewed submitted code
[x] I added tests to verify changes
[x] No new PII added or SDK only sends newly added PII if sendDefaultPii is enabled
[ ] I updated the docs if needed
[x] All tests passing
[x] No breaking changes

:crystal_ball: Next steps

github-actions[bot] commented 2 weeks ago

🚨 Detected changes in high risk code 🚨

High-risk code has higher potential to break the SDK and may be hard to test. To prevent severe bugs, apply the rollout process for releasing such changes and be extra careful when changing and reviewing these files:

[x] flutter/lib/src/integrations/native_app_start_integration.dart

github-actions[bot] commented 2 weeks ago

	Messages
:book:	Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by :no_entry_sign: dangerJS against f993041e490a27cd2ee2f55d3cb3d16668e3e657

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.

Project coverage is 86.05%. Comparing base (404f5a9) to head (f993041). Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
flutter/lib/src/frame_callback_handler.dart	50.00%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #2405 +/- ## ========================================== + Coverage 85.04% 86.05% +1.00% ========================================== Files 257 81 -176 Lines 9194 2875 -6319 ========================================== - Hits 7819 2474 -5345 + Misses 1375 401 -974 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

github-actions[bot] commented 2 weeks ago

Android Performance metrics :rocket:

	Plain	With Sentry	Diff
Startup time	459.28 ms	540.31 ms	81.02 ms
Size	6.49 MiB	7.56 MiB	1.07 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
453e1bca2bc18a0a639cc7f796ed58c7c44d89b7	320.41 ms	372.73 ms	52.32 ms
061fed2925fdc038a8b4ed8f7905e646d7c0a887	434.11 ms	506.49 ms	72.38 ms
9928a7433f1f08e0a8a85af66aa00fe0f3757472	375.26 ms	456.30 ms	81.04 ms
11fb40841ad3d6fc46e02eb8adeb74d5b6932582	320.10 ms	380.24 ms	60.14 ms
dd76eef96d93031410fb154fb4bd51b57d36854b	461.37 ms	540.55 ms	79.18 ms
2261c15342b55a1566de274be3688b1e34743749	370.00 ms	455.88 ms	85.88 ms
d089990aa16ee19071abb70d6bab8231c452fc13	361.67 ms	442.50 ms	80.83 ms
ca7f531d8ef46d53d71b084884c27cae425de945	395.69 ms	497.82 ms	102.12 ms
e5b744f308ef88a70f679a622a88ade69705613c	302.70 ms	342.17 ms	39.47 ms
1e094d3e5f001e17916563b01d9e85ffc2baffa4	316.22 ms	378.69 ms	62.47 ms

App size

Revision	Plain	With Sentry	Diff
453e1bca2bc18a0a639cc7f796ed58c7c44d89b7	5.94 MiB	6.95 MiB	1.01 MiB
061fed2925fdc038a8b4ed8f7905e646d7c0a887	6.52 MiB	7.59 MiB	1.06 MiB
9928a7433f1f08e0a8a85af66aa00fe0f3757472	5.94 MiB	6.96 MiB	1.02 MiB
11fb40841ad3d6fc46e02eb8adeb74d5b6932582	6.06 MiB	7.10 MiB	1.04 MiB
dd76eef96d93031410fb154fb4bd51b57d36854b	6.35 MiB	7.40 MiB	1.05 MiB
2261c15342b55a1566de274be3688b1e34743749	6.27 MiB	7.20 MiB	957.75 KiB
d089990aa16ee19071abb70d6bab8231c452fc13	6.34 MiB	7.28 MiB	967.79 KiB
ca7f531d8ef46d53d71b084884c27cae425de945	6.33 MiB	7.26 MiB	949.75 KiB
e5b744f308ef88a70f679a622a88ade69705613c	6.06 MiB	7.09 MiB	1.03 MiB
1e094d3e5f001e17916563b01d9e85ffc2baffa4	6.16 MiB	7.14 MiB	1004.21 KiB

Previous results on branch: enh/app-start-time

Startup times

Revision	Plain	With Sentry	Diff
21acda16ea089c41ad2fda0e8e3bf3c5531cd7ec	450.35 ms	492.31 ms	41.96 ms
92b5b84f3a732b6c2e0842b31379cb985f621e25	453.60 ms	485.92 ms	32.32 ms
59905627261ca2ec5bc8efe0b2365ef9d64dc282	463.09 ms	508.14 ms	45.05 ms
ca24b538699aecd462f2b8bfb782c2d45577c0b3	475.92 ms	551.68 ms	75.76 ms
b1e9b9cb3930fb0d6203597cb9d93938099447c6	469.20 ms	500.98 ms	31.78 ms
803684be5c9d2e567ca871f39bec8b18b03fa50d	455.88 ms	497.16 ms	41.29 ms
d724aacfdf7bd33dbb6132a56de2f43d28fb0188	482.20 ms	512.35 ms	30.14 ms
4def0cef66500e4d7a5c00f4a7295714045fff9e	465.86 ms	494.74 ms	28.88 ms
9be48296fe261a89fabc041a3574202ac6363b5d	438.42 ms	458.45 ms	20.03 ms
88bc8a5c757eeec4658a1b6690d3d872610a2516	482.94 ms	515.82 ms	32.88 ms

App size

Revision	Plain	With Sentry	Diff
21acda16ea089c41ad2fda0e8e3bf3c5531cd7ec	6.49 MiB	7.56 MiB	1.07 MiB
92b5b84f3a732b6c2e0842b31379cb985f621e25	6.49 MiB	7.56 MiB	1.07 MiB
59905627261ca2ec5bc8efe0b2365ef9d64dc282	6.49 MiB	7.56 MiB	1.07 MiB
ca24b538699aecd462f2b8bfb782c2d45577c0b3	6.49 MiB	7.56 MiB	1.07 MiB
b1e9b9cb3930fb0d6203597cb9d93938099447c6	6.49 MiB	7.56 MiB	1.07 MiB
803684be5c9d2e567ca871f39bec8b18b03fa50d	6.49 MiB	7.56 MiB	1.07 MiB
d724aacfdf7bd33dbb6132a56de2f43d28fb0188	6.49 MiB	7.56 MiB	1.07 MiB
4def0cef66500e4d7a5c00f4a7295714045fff9e	6.49 MiB	7.56 MiB	1.07 MiB
9be48296fe261a89fabc041a3574202ac6363b5d	6.49 MiB	7.56 MiB	1.07 MiB
88bc8a5c757eeec4658a1b6690d3d872610a2516	6.49 MiB	7.56 MiB	1.07 MiB

buenaflor commented 2 weeks ago

cc @stefanosiano @denrase please take a look

github-actions[bot] commented 1 week ago

iOS Performance metrics :rocket:

	Plain	With Sentry	Diff
Startup time	1254.24 ms	1274.58 ms	20.34 ms
Size	8.38 MiB	9.77 MiB	1.40 MiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
824df587e3e4f7db4f5934277323564a46a5e30d	1235.72 ms	1259.02 ms	23.30 ms
3f3ef0be1de5e508cf4723d8ea213dcc1b7751f8	1223.73 ms	1237.67 ms	13.94 ms
4c78360ef218251d46e8468c8aaf00132482e4a4	1230.35 ms	1252.37 ms	22.01 ms
ca9f3989247d3e0d5828e2c3ca730fb8a5182bea	1227.29 ms	1236.20 ms	8.92 ms
dd933d4e20ee057899c79921f0f9b980425e4041	1238.73 ms	1252.43 ms	13.70 ms
afa6e2a6ded2f7dbb4530fe3974fdbb30db1f914	1251.04 ms	1266.65 ms	15.61 ms
a49594a2366317ecfbf3078f2573c410cc51ad0b	1284.83 ms	1313.29 ms	28.45 ms
a094100fbe83db65d496add0d611039773ca66a5	1260.27 ms	1276.75 ms	16.48 ms
8fa393466a5d478f281e46cbc11e66c2b3db12d8	1252.79 ms	1272.41 ms	19.62 ms
613760b51612c5874f586dd29349e387b289c8cd	1263.10 ms	1277.27 ms	14.16 ms

App size

Revision	Plain	With Sentry	Diff
824df587e3e4f7db4f5934277323564a46a5e30d	8.33 MiB	9.64 MiB	1.31 MiB
3f3ef0be1de5e508cf4723d8ea213dcc1b7751f8	8.32 MiB	9.38 MiB	1.05 MiB
4c78360ef218251d46e8468c8aaf00132482e4a4	8.32 MiB	9.38 MiB	1.06 MiB
ca9f3989247d3e0d5828e2c3ca730fb8a5182bea	8.38 MiB	9.74 MiB	1.36 MiB
dd933d4e20ee057899c79921f0f9b980425e4041	8.33 MiB	9.64 MiB	1.31 MiB
afa6e2a6ded2f7dbb4530fe3974fdbb30db1f914	8.28 MiB	9.33 MiB	1.05 MiB
a49594a2366317ecfbf3078f2573c410cc51ad0b	8.16 MiB	9.16 MiB	1.00 MiB
a094100fbe83db65d496add0d611039773ca66a5	8.16 MiB	9.17 MiB	1.01 MiB
8fa393466a5d478f281e46cbc11e66c2b3db12d8	8.09 MiB	9.07 MiB	1000.86 KiB
613760b51612c5874f586dd29349e387b289c8cd	8.15 MiB	9.13 MiB	1000.46 KiB

Previous results on branch: enh/app-start-time

Startup times

Revision	Plain	With Sentry	Diff
d724aacfdf7bd33dbb6132a56de2f43d28fb0188	1246.35 ms	1271.84 ms	25.48 ms
9be48296fe261a89fabc041a3574202ac6363b5d	1224.19 ms	1243.20 ms	19.01 ms
b1e9b9cb3930fb0d6203597cb9d93938099447c6	1252.49 ms	1273.52 ms	21.03 ms
803684be5c9d2e567ca871f39bec8b18b03fa50d	1253.82 ms	1263.78 ms	9.96 ms
34988b19d263bafa712f558d270a044915a3c5a9	1252.16 ms	1268.30 ms	16.13 ms
59905627261ca2ec5bc8efe0b2365ef9d64dc282	1245.67 ms	1275.57 ms	29.90 ms
4def0cef66500e4d7a5c00f4a7295714045fff9e	1252.67 ms	1274.83 ms	22.16 ms
1fec51e142868b8778e3d5441aa874a9e9c22ea3	1243.58 ms	1275.52 ms	31.94 ms
92b5b84f3a732b6c2e0842b31379cb985f621e25	1233.21 ms	1244.96 ms	11.75 ms
88bc8a5c757eeec4658a1b6690d3d872610a2516	1242.29 ms	1264.04 ms	21.75 ms

App size

Revision	Plain	With Sentry	Diff
d724aacfdf7bd33dbb6132a56de2f43d28fb0188	8.38 MiB	9.77 MiB	1.40 MiB
9be48296fe261a89fabc041a3574202ac6363b5d	8.38 MiB	9.77 MiB	1.40 MiB
b1e9b9cb3930fb0d6203597cb9d93938099447c6	8.38 MiB	9.77 MiB	1.40 MiB
803684be5c9d2e567ca871f39bec8b18b03fa50d	8.38 MiB	9.77 MiB	1.40 MiB
34988b19d263bafa712f558d270a044915a3c5a9	8.38 MiB	9.77 MiB	1.40 MiB
59905627261ca2ec5bc8efe0b2365ef9d64dc282	8.38 MiB	9.77 MiB	1.40 MiB
4def0cef66500e4d7a5c00f4a7295714045fff9e	8.38 MiB	9.77 MiB	1.40 MiB
1fec51e142868b8778e3d5441aa874a9e9c22ea3	8.38 MiB	9.77 MiB	1.40 MiB
92b5b84f3a732b6c2e0842b31379cb985f621e25	8.38 MiB	9.77 MiB	1.40 MiB
88bc8a5c757eeec4658a1b6690d3d872610a2516	8.38 MiB	9.77 MiB	1.40 MiB

getsentry / sentry-dart