facebook / screenshot-tests-for-android

Generate fast deterministic screenshots during Android instrumentation tests
http://facebook.github.io/screenshot-tests-for-android
Apache License 2.0
1.74k stars 229 forks source link

test-run-id folder not created for screenshots -> No such file or directory #293

Closed stetro closed 3 years ago

stetro commented 3 years ago

I'm trying to get this extension running but facing an issue after the tests run through. I can see the created screenshots in /sdcard/screenshots/...packagename.../screenshots-default/ but the python scripts tries to pull from /sdcard/screenshots/...packagename.../screenshots-default/59b35fe9-f3bb-4b20-806e-a1f3b4a686aa which in fact is a non existing folder.

> Task :app:recordDebugAndroidTestScreenshotTest FAILED
[-m, android_screenshot_tests.pull_screenshots, --apk, /...path.../app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk, --test-run-id, 59b35fe9-f3bb-4b20-806e-a1f3b4a686aa, --temp-dir, /...path.../app/build/screenshotsDebugAndroidTest, --record, screenshots]
/sdcard/screenshots/...packagename.../screenshots-default/metadata.json: 1 file pulled, 0 skipped. 0.7 MB/s (3354 bytes in 0.005s)
ls: /sdcard/screenshots/...packagename.../screenshots-default/59b35fe9-f3bb-4b20-806e-a1f3b4a686aa: No such file or directory

Any idea what I'm missing here?

stetro commented 3 years ago

Found the solution myself - turned out that the test-run-id was not passed to the TestRunner because I have overwritten it with my instrumented test configuration:

// only run screenshot instrumented tests when called and vice versa
if (project.gradle.startParameter.taskNames.any { it.endsWith("ScreenshotTest") }) {
    setTestInstrumentationRunnerArguments(["package": "..packagename...screenshot"])
} else {
    setTestInstrumentationRunnerArguments(["notPackage": "..packagename...screenshot"])
}

Insted of overwriting the arguments they can be extended like this:

// only run screenshot instrumented tests when called and vice versa
if (project.gradle.startParameter.taskNames.any { it.endsWith("ScreenshotTest") }) {
    testInstrumentationRunnerArguments["package"] = "..packagename...screenshot"
} else {
    testInstrumentationRunnerArguments["notPackage"] = "..packagename...screenshot"
}

May this is helpful for someone else :)