takahirom / roborazzi

Make JVM Android integration test visible 🤖📸
https://takahirom.github.io/roborazzi/
Apache License 2.0
695 stars 28 forks source link

Having separate directories for recording vs verification #471

Open shubhamgarg1 opened 3 weeks ago

shubhamgarg1 commented 3 weeks ago

We have started using Roborazzi in one of our projects and find it to be really useful. We store our recorded screenshots in a directory which is tracked by git intests/resources directory. But on CI, we change the output directory to one inside the build directory. We change directories based on whether we are running in record mode or verification mode. I think it would be helpful if such support is provided by Roborazzi itself where one can supply the recordingDir and a validationDir to control this behavior.

takahirom commented 3 weeks ago

Thank you for reporting this issue. I'm not certain if this addresses your needs, but we have a feature called roborazziOptions.compareOptions.outputDirectoryPath that allows you to save comparison files separately. Would you mind trying it out? Here's the relevant link to the source code: https://github.com/takahirom/roborazzi/blob/4e2297768c45474357e136b64568780ffde000fd/include-build/roborazzi-core/src/commonJvmMain/kotlin/com/github/takahirom/roborazzi/RoborazziOptions.kt#L100

shubhamgarg1 commented 2 weeks ago

Yes, we use outputDirectoryPath and change it based on whether we are running with recording or with validations. But it would be helpful to have separate declarations for both. E.g., locally, one would want to run a recording with a path that is inside some folder that can be committed to git. But validations are more likely to happen in a path inside a build directory and that can be ignored by git.

shubhamgarg1 commented 2 weeks ago

We currently use a configuration like this to achieve the above:


tasks.withType(Test) {
    dependsOn('copyRoborazziScreenshotsFromRecordingToVerificationDirectory')
}

// Setting different Roborazzi directories.
// We have done this so as to not pollute the recording directory in case someone is running validations.
// For more information, refer to ScreenshotTesting.md
final String ROBORAZZI_OUTPUT_DIRECTORY_VERIFICATION = "build/reports/roborazzi/resources/screenshots"
final String ROBORAZZI_OUTPUT_DIRECTORY_RECORDING = "src/test/resources/screenshots/roborazzi"

roborazzi {
    if (getProperty("roborazzi.test.record").equalsIgnoreCase("true")) {
        // In case of recording, use the directory where the screenshots are stored.
        outputDir = layout.projectDirectory.dir(ROBORAZZI_OUTPUT_DIRECTORY_RECORDING)
    }
    else if (getProperty("roborazzi.test.verify").equalsIgnoreCase("true")) {
        // In case of verification, use the directory where the screenshots are validated from..
        outputDir = layout.projectDirectory.dir(ROBORAZZI_OUTPUT_DIRECTORY_VERIFICATION)
    }
    // Else, let Roborazzi use the default directory which is build/outputs/roborazzi.
}

tasks.register('copyRoborazziScreenshotsFromRecordingToVerificationDirectory') {
    doLast {
        if (getProperty("roborazzi.test.verify").equalsIgnoreCase("true")) {
            copy {
                from ROBORAZZI_OUTPUT_DIRECTORY_RECORDING
                into ROBORAZZI_OUTPUT_DIRECTORY_VERIFICATION
            }
        }
    }
}