Executing ExampleStartupBenchmark with or without baseline-prof.txt does not produce very different results

[goldorak84]

I went through all the steps to generate a baseline profile with the gradle virtual device. Running ExampleStartupBenchmark using the generated baseline_prof.txt, I had results similar to the codelab's. Results about 5% less than with the None compilation mode.

I wanted to do the same test, but without a baseline profile. So I deleted the baseline profile, deleted my app build folder, and ran the test again. I got really similar results than with the baseline profile.

Do you have any idea on why this is happening?

[mlykotom]

We had some troubles in past with pre-installed code on the device on a non-rooted build.

Could you try with the latest versions of the profile installer and macrobenchmark?

Alternatively, would it be feasible to measure the results on a userdebug (rooted) device? On rooted devices, we can properly clear the app state between the benchmarks, which results in good measurements.

In any case, this only affects measuring the results, not the Baseline Profiles directly. Alternatively, you can upload the APK with generate Baseline Profiles to Google Play and you should see the benefits in Play Vitals or any other service you use for measuring app startup or jank

[mlykotom]

I'm going to close this due to no activity, but feel free to reopen if you still see no improvements when using Baseline Profiles. Thanks! 🤗