Cloud function unable to find the generated .json file

[avidraghav]

I have a little confusion: What should be the environment variable which specifies target package? In the sample workflow provided under ftl package, the target package environment variable is set to com.example.macrobenchmark which is the package name of the macrobenchmark module which contains test classes not the app module which contains classes that have to be benchmarked.

One has to generate 2 apks as per documentation, one which is as close to release build as possible (init with release and signingconfig = debug) and other apk is the benchmark apk. I'm doing this in my CI by :app:assemble[VariantName] and :macrobenchmark:assemble[VariantName] respectively.

say my app has package com.raghav and the benchmark apk has the package name com.raghav.benchmark

Which of these has to be used in the package name environment variable? Also while configuring CI action for running tests what apk path has to be provided for --app and--test and it is mentioned to provide the path to external storage to store and retrieve the json file, if i'm not providing it the json file is not getting generated at all if am I providing it my cloud function is not looking for that file inside the provided path and giving the error 'Unable to locate the object'.

The main reason I find for the whole confusion is that the sample images provided in the readme under ftl package are not matching with the contents of that file. Thanks in advance :)

[mlykotom]

Hi, thanks for the question!

Assuming your app has package com.raghav and benchmark has com.raghav.benchmark, then:

1) You set measureRepeated(packageName="com.raghav", ...) from your benchmarks. This way the benchmarks work with the app package.

2) The benchmarks are represented in the form of test apk, therefore the parameters are: --app /YourApp/app/build/outputs/apk/benchmark/app-benchmark.apk (this is the output after running :app:assembleBenchmark) --test /YourApp/macrobenchmark/build/outputs/apk/benchmark/macrobenchmark-benchmark.apk (this is the output after running :macrobenchmark:assembleBenchmark)

3) To store and retrieve the JSON file, you need to select external storage path that is writable by the application. I'd recommend using /sdcard/Download that should be writable even with scoped storage. So in this case you should add it to the environment-variables param: --environment-variables clearPackageData=true,additionalTestOutputDir=/sdcard/Download,no-isolated-storage=true

And then you can pull the folder with --directories-to-pull /sdcard/Download

This is the whole command we use in actions:

  gcloud firebase test android run \
            --type instrumentation \
            --app ${{ github.workspace }}/MacrobenchmarkSample/app/build/outputs/apk/benchmark/app-benchmark.apk \
            --test ${{ github.workspace }}/MacrobenchmarkSample/macrobenchmark/build/outputs/apk/benchmark/macrobenchmark-benchmark.apk \
            --device model=redfin,version=30,locale=en,orientation=portrait \
            --directories-to-pull /sdcard/Download \
            --results-bucket gs://macrobenchmark-results \
            --environment-variables clearPackageData=true,additionalTestOutputDir=/sdcard/Download,no-isolated-storage=true \
            --timeout 30m

[avidraghav]

Hi, thanks for the reply but turns out I'm doing just the same thing. I'll go through my CI configuration once again and will update here if I get the desired result.