Error while processing screenshot testing result

[olegosipenko]

Expected behaviour

debugExecuteScreenshotTests should record screenshot tests, pull them from device and verify against the reference.

Actual behaviour

After upgrading to the latest (5.12.2) version we started to receive the following exception while running screenshot tests:

> Task :library-module:connectedDebugAndroidTest
Starting 26 tests on Snapshotter(AVD) - 9

> Task :library-module:debugDownloadScreenshots
⬇️  Pulling screenshots from your connected devices!
Shot ADB warning: We could not pull screenshots from folder: /storage/emulated/0/Download/screenshots/com.app.test.uid/screenshots-default/
Shot ADB warning: rm: /storage/emulated/0/Download/screenshots/com.app.test.uid/screenshots-default/: No such file or directory

> Task :library-module:debugExecuteScreenshotTests FAILED
🔎  Comparing screenshots with previous ones.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':library-module:debugExecuteScreenshotTests'.
> com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;

We had to upgrade the version, because when we tried to migrate our screenshot test suite to use Compose, we started to receive the error mentioned in #244 , which was strange, because we are running this under Mac os, on emulator with SDK v28.

> Task :library-module:connectedDebugAndroidTest
Starting 26 tests on Snapshotter(AVD) - 9

> Task :library-module:debugDownloadScreenshots
⬇️  Pulling screenshots from your connected devices!
Shot ADB warning: We could not pull screenshots from folder: /storage/emulated/0/Download/screenshots/com.app.test.uid/screenshots-default/
Shot ADB warning: rm: /storage/emulated/0/Download/screenshots/com.app.test.uid/screenshots-default/: No such file or directory

> Task :library-module:debugExecuteScreenshotTests FAILED
🔎  Comparing screenshots with previous ones.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':library-module:debugExecuteScreenshotTests'.
> org.apache.commons.io.FileExistsException: Destination '/Users/user/StudioProjects/app/library-module/screenshots/debug/screenshots-default/com.app.test.uid.presentation.fragment.tool.elements.StepTest_testStepWithFallbackDescriptionPrimaryButton.png' already exists

Steps to reproduce

• Create screenshot tests for Compose in library module
• try to run debugExecuteScreenshotTests
• observe error

Version of the library

5.12.2 Mac OS 11.5.1 Java /Library/Java/JavaVirtualMachines/jdk-11.0.11.jdk/Contents/Home

BTW Previously plugin saved screenshots in the folder $module/screenshots/$flavor/ now after execution I see screenshots-default folder inside previously existing folder. And for my 55 tests were added 705 new files in this screenshots-default folder - one file metadata.xml_emulator-5554, 54 JSONs - like this com.roadrunner.automationtool.presentation.fragment.automationtool.elements.ButtonTest_testDropdownButtonDisabled_dump.json, 648 PNGs and one screenshot_bundle.zip with all this PNGs and JSONs. Is it ok?

[pedrovgs]

Hi @olegosipenko could you please try to cone this repository and run the tests you can find in shot-consumer-compose?

[olegosipenko]

Hi @olegosipenko could you please try to cone this repository and run the tests you can find in shot-consumer-compose?

Sure, will update when done

[olegosipenko]

Ok, so running tests from shot-consumer-compose finishes successfully. BTW tried to update Shot version in our project to 5.13.0 and still got

> com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;

[zoey-juan]

the same issue is discussing in #244