mapbox / mapbox-navigation-android

Mapbox Navigation SDK for Android
https://docs.mapbox.com/android/navigation/overview/
Other
622 stars 319 forks source link

Add abilty to upload history files to circleci artifacts #4506

Open kmadsen opened 3 years ago

kmadsen commented 3 years ago

~In the instrumentation tests we could add a MapboxHistoryTestRule~ A TestWatcher creates lifecycle issues with MapboxNavigation. Update to a test object, TestMapboxHistoryRecorder .

That test rule can save the history files from any test or benchmark. I think the file will need to be stored in external storage. That way we can move upload it to artifacts. Similar solution has been made for mobile-metrics.

Specifically looking to do this to help us solve this one https://github.com/mapbox/mapbox-navigation-android/pull/4419

kmadsen commented 3 years ago

As @Guardiola31337 found, we don't want to do this with a TestWatcher. The reason is, because of the lifecycle scopes created by test runners. The testing utility class should explicitly save the file. That way we can call it before we handle MapboxNavigation.onDestory in the @After test block

https://github.com/mapbox/mapbox-navigation-android/pull/4516#discussion_r653575232

kmadsen commented 3 years ago

@Guardiola31337 has a example of this working on a branch https://github.com/mapbox/mapbox-navigation-android/pull/4726

  1. MapboxHistoryTestRule will write history files to the /sdcard/Download/mapbox_test device directory
  2. gcloud firebase test android run --type instrumentation ... --directories-to-pull=/sdcard/Download/mapbox_test \
  3. Take a test run https://console.firebase.google.com/u/0/project/mapbox-navigation-android/testlab/histories/bh.54eb2dd57e959221/matrices/5268338467094668976/executions/bs.9376f2a7cee4a1f2
  4. Click on Test Results https://console.cloud.google.com/storage/browser/test-lab-052kt7ra614ai-hjrc98wu2tksc/2021-08-24_20:06:02.084320_QKPX/hammerhead-23-fr-portrait?authuser=0
  5. Find the test results in the artifact directory https://console.cloud.google.com/storage/browser/test-lab-052kt7ra614ai-hjrc98wu2tksc/2021-08-24_20:06:02.084320_QKPX/hammerhead-23-fr-portrait/artifacts?authuser=0&pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&prefix=&forceOnObjectsSortingFiltering=false