takahirom / roborazzi

Make JVM Android integration test visible 🤖📸
https://takahirom.github.io/roborazzi/
Apache License 2.0
649 stars 24 forks source link

Declare input directories as files to avoid issues when they don't exist #374

Closed lukas-mercari closed 1 month ago

lukas-mercari commented 1 month ago

I sometimes get this error:

* What went wrong:
A problem was found with the configuration of task ':sample-android:testDebugUnitTest' (type 'AndroidUnitTest').
  - Type 'com.android.build.gradle.tasks.factory.AndroidUnitTest' property '$1' specifies directory '/Users/lukas/Dev/test/roborazzi/sample-android/build/outputs/roborazzi' which doesn't exist.

    Reason: An input file was expected to be present but it doesn't exist.

    Possible solutions:
      1. Make sure the directory exists before the task is called.
      2. Make sure that the task which produces the directory is declared as an input.

    For more information, please refer to https://docs.gradle.org/8.4/userguide/validation_problems.html#input_file_does_not_exist in the Gradle documentation.

It's a bit hard to reproduce, since I believe the configuration cache needs to kick in, but the following steps usually work:

// Make a change to the tests in `sample-android`.
./gradlew :sample-android:recordRoborazziDebug --build-cache
rm -rf ./sample-android/build
./gradlew :sample-android:recordRoborazziDebug --build-cache
rm -rf ./sample-android/build
./gradlew :sample-android:recordRoborazziDebug --build-cache

I believe the issue is essentially what is described in https://github.com/gradle/gradle/issues/2016.

takahirom commented 1 month ago

Thank you for your investigation. I believe the contents of the output directory are changed during the build process, causing the test to fail.

https://github.com/takahirom/roborazzi/blob/7e8d4f92dfffcf062accd487038624ab21d7cd44/include-build/roborazzi-gradle-plugin/src/main/java/io/github/takahirom/roborazzi/RoborazziPlugin.kt#L109

checkIfOutputIsUsed java.lang.AssertionError: Expected count: 0, actual count: 1 summary:ResultSummary(total=1, recorded=0, added=0, changed=1, unchanged=0) at io.github.takahirom.roborazzi.RoborazziGradleProject.checkResultCount(RoborazziGradleProject.kt:327) at io.github.takahirom.roborazzi.RoborazziGradleProject.checkResultCount$default(RoborazziGradleProject.kt:312) at io.github.takahirom.roborazzi.RoborazziGradleProjectTest.checkIfOutputIsUsed(RoborazziGradleProjectTest.kt:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

https://github.com/takahirom/roborazzi/blob/main/include-build/roborazzi-gradle-plugin/src/integrationTest/java/io/github/takahirom/roborazzi/RoborazziGradleProjectTest.kt#L204

lukas-mercari commented 1 month ago

@takahirom Sorry, I ran out of time to finish this today. Unfortunately I don't have a good idea on how gradle works, so all changes take me a bit of time. 😄 Let me double check the behaviour in the next days, possibly also in combination with what we conclude in #363.

Judging from my local runs it works for resolving the error I mentioned in the description -- and thus I can reproduce the issue that #366 fixes again.

takahirom commented 1 month ago

@lukas-mercari Thanks! Let me know if you think we should release these PRs. https://github.com/takahirom/roborazzi/pull/374 https://github.com/takahirom/roborazzi/pull/366

takahirom commented 1 month ago

@lukas-mercari If you feel like we're getting stuck, I think it's okay to release these changes as they are. I'd rather improve things gradually than aim for perfection. What do you think? If you need more time, that's fine too.

lukas-mercari commented 1 month ago

@takahirom Sorry, it got delayed a bit since I went travelling! 🙇 Indeed I think it'd make sense to merge the two PRs.

takahirom commented 1 month ago

Thanks!