Cannot save screenshots on Windows, works on MACos

GustavoEliseu commented 1 year ago

Preface

I am integrating screenshot testing in my Jetpack Compose application. I'm using api level 34 and i'm facing some issues that only affect my Windows computer, on the same project running on my MacOS works flawless. Already using the non-SDK interfaces https://github.com/pedrovgs/Shot#using-shot-on-api-28

Dependencies

Shot => 6.0.0 Jetpack Compose => 1.5.3 Android Gradle Plugin => 8.1.2

Systems

Failure on this system:

System Windows 11 Android Studio Giraffe 2022.3.1 ProjectPath

C:\Users\gusta\AndroidStudioProjects\Mypokedex

Working on this system:

System MacOS Ventura 13.6 Android Studio Giraffe 2022.3.1 ProjectPath

/Users/Amaral/Documents/GitHub/MyPokedexApp

Configuration


plugins {
    alias(libs.plugins.com.android.library)
    alias(libs.plugins.org.jetbrains.kotlin.android)
    id("kotlin-kapt")
    id("shot")
}

android {
    namespace = "com.gustavoeliseu.pokedexlist"
    compileSdk = 34

    shot {
        applicationId = "ge.pokedexlist"
    }
    defaultConfig {
        testApplicationId = "ge.pokedexlist"
        minSdk = 26
        testInstrumentationRunner = "com.karumi.shot.ShotTestRunner"
    }

    buildFeatures {
        compose = true
    }
    composeOptions {
        kotlinCompilerExtensionVersion = "1.4.7"
    }

Sample

 @get:Rule
    var composeRule = createComposeRule()

    @Test
    fun verify_pokemon_ditto_card_snapshot() {
        composeRule.setContent {
            PokemonCardTest(
                id = 132,
                name = "Ditto",
                pokemonColorId = 6
            )
        }
        compareScreenshot(composeRule)
    }

Device

Tested it with 2 emulators with Pixel 6 API 349 and API 26 via Android Studio. Also tested with a Xiamoi poco f3 (M2012K11AG). Got the same result with all of them.

Command

./gradlew executeScreenshotTests -Precord

Output

> Task :app:debugDownloadScreenshots
??  Pulling screenshots from your connected devices!
Shot ADB warning: We could not pull screenshots from folder: /storage/emulated/0/Download/screenshots/ge.pokedexlist/screenshots-default/
Shot ADB warning: We could not pull screenshots from folder: /storage/emulated/0/Download/screenshots/ge.pokedexlist/screenshots-compose-default/
Shot ADB warning: rm: /storage/emulated/0/Download/screenshots/ge.pokedexlist/screenshots-default/: No such file or directory
Shot ADB warning: rm: /storage/emulated/0/Download/screenshots/ge.pokedexlist/screenshots-compose-default/: No such file or directory

> Task :app:debugExecuteScreenshotTests
?  Saving screenshots.
? We couldn't find any screenshot. Did you configure Shot properly and added your tests to your project? https://github.com/pedrovgs/Shot/#getting-started

Behaviour

The screenshots are being generated and are deleted when the command ends, but only on windows it fails when trying to copy them to the project folder. Maybe it's related to the saving path on windows.

I never worked with Scala but if you give me a direction to where look at, i can run some tests on both machines to find the cause of this problem. I can also create a dualboot with linux to test on all platforms.

GustavoEliseu commented 1 year ago

Did a few tests after forking the repository, the problem seems to be simple to solve, it's related to the last '\' on the path. I believe in windows it's not supposed to have the last '\'

original:

C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\screenshots\pro\debug\

how it worked on windows:

C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\screenshots\pro\debug

When i hardcoded without that last '\' the files started to save on windows, but now i'm facing a new error related to moving and comparing the screenshots. Probably also related to the last '\'.

That's the problem on Windows, but i'm finding it dificult to fix without breaking any existing code. If anyone more experienced with this code could take a look it would be a lot faster.

GustavoEliseu commented 1 year ago

Did a few more tests, and for some reason on windows you need to remove the '\' when creating a folder, but that '\' is needed when deleting the same folder. There's still one huge problem, but it's related to another issue.

Adb.scala

  private def executeAdbCommand(command: String): Int = {
    var newCommand = s"${adbPath} $command"
    if (newCommand.endsWith("/")) newCommand = newCommand.dropRight(1)
    if(newCommand.endsWith("\\"))newCommand= newCommand.dropRight(1)
    s"$newCommand" ! logger
  }

  private def executeAdbCommandWithResult(command: String): String = {
    var newCommand = s"${adbPath} $command"
    if (newCommand.endsWith("/")) newCommand = newCommand.dropRight(1)
    if (newCommand.endsWith("\\")) newCommand = newCommand.dropRight(1)
    s"$newCommand" !! logger
  }

That solved the copy problem on windows, but there's still the problem of not being able to delete "metadata.json" and "com.karumi.shotconsumercompose.MainActivityTest_activityTest_dump.json" due to them being used. When you try to run it again, it would lead to an error with metadata.json

Shot.scala

  private def safeDeleteDirectory(file: File): Unit = {
    var fileToDelete = file
    try {
      if(!fileToDelete.exists() && !fileToDelete.getAbsolutePath.endsWith("\\")) {
        fileToDelete = new File(file.getAbsolutePath+"\\")
      }
      FileUtils.deleteDirectory(fileToDelete)
    } catch {
      case e: Throwable => {
        println(YELLOW + s"Failed to delete directory: ${e}")
        println(YELLOW + s"Failed to delete directory: ${e.getCause.getCause}")
      }
    }
  }

Error when running safeDeleteDirectory

??  Pulling screenshots from your connected devices!

> Task :app:proDebugExecuteScreenshotTests                                                                                                                                                                                              
?  Saving screenshots.
?  Screenshots recorded and saved at: C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\screenshots\pro\debug\
?  You can review the execution report here: C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\build\reports\shot\pro\debug\index.html
Failed to delete directory: org.apache.commons.io.IOExceptionList: C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\screenshots\pro\debug\screenshots-default
Failed to delete directory: java.nio.file.FileSystemException: C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\screenshots\pro\debug\screenshots-default\com.karumi.shotconsumercompose.MainActivityTest_activityTest_dump.json: The process cannot access the file because it is being used by another process.

Running a second time after the error

 org.apache.commons.io.FileExistsException: File element in parameter 'null' already exists: 'C:\Users\gusta\Documents\GitHub\Shot\shot-consumer-compose\app\screenshots\pro\debug\screenshots-default\metadata.json_3297ed05'

soyo2 commented 6 months ago

Same issue here unfortunately. Works fine on MACs, but not on Windows. And it's not only saving screenshots, it's running the tests themselves to compare and verify also suffer from the same problem ("? We couldn't find any screenshot. Did you configure Shot properly and added your tests to your project?")

pedrovgs / Shot