facebook / screenshot-tests-for-android

Generate fast deterministic screenshots during Android instrumentation tests
http://facebook.github.io/screenshot-tests-for-android
Apache License 2.0
1.74k stars 229 forks source link

API 17: java.lang.RuntimeException: Failed to create the directory for screenshots. Is your sdcard directory read-only? #162

Open vanniktech opened 6 years ago

vanniktech commented 6 years ago

Executing the screenshots with connectedTest fails on an emulator with SDK 17 and succeeds at SDK 21.

Repro steps:

Create an emulator with SDK 17

echo no | avdmanager create avd --force -n test -k "system-images;android-17;default;armeabi-v7a"
mksdcard -l e 512M test-sd.img # Needed for Facebooks screenshot testing.
$ANDROID_HOME/emulator/emulator -avd test -sdcard test-sd.img -no-audio -no-window &

execute connectedCheck task and it'll fail.

When using this setup it works without any problems:

echo no | avdmanager create avd --force -n test -k "system-images;android-21;default;armeabi-v7a"
mksdcard -l e 512M test-sd.img # Needed for Facebooks screenshot testing.
$ANDROID_HOME/emulator/emulator -avd test -sdcard test-sd.img -no-audio -no-window &
xiphirx commented 6 years ago

Hmm interesting. I'll have to look into why this is happening.

vanniktech commented 6 years ago

Any news yet? Or anything I can do to assist?

xiphirx commented 6 years ago

Sorry, I haven't had time to look into this yet, currently swamped with other things :( . If you have the time, debugging the issue would be very helpful. I'm unsure why we dont have permission to write to external storage on API 17...

xiphirx commented 6 years ago

Still haven't spent enough time debugging this, however if you can run some adb commands prior to running your tests, then you should be able to the following:

$ adb -e shell
$ su
$ mount -o rw,remount rootfs /
$ chmod 777 /mnt/sdcard
$ exit

This will remount your external storage to have write capability, and should work around the issue while I try to find time to figure out whats going wrong.

xiphirx commented 6 years ago

Its been a while since this task was updated, and the workaround I posted should alleviate this issue. Please comment here if it doesnt.

vanniktech commented 6 years ago

Sorry this went off my radar. I've tried the workaround but I'm still getting the same exception.

fespinoza commented 5 years ago

I also have this error... minSdkVersion 21 and targetSdkVersion 28, with 0.8.0

java.lang.RuntimeException: Failed to create the directory /sdcard/screenshots/com.schibsted.peil.android.dev.test/screenshots-default for screenshots. Is your sdcard directory read-only?
at com.facebook.testing.screenshot.internal.ScreenshotDirectories.getSdcardDir(ScreenshotDirectories.java:121)
at com.facebook.testing.screenshot.internal.ScreenshotDirectories.get(ScreenshotDirectories.java:53)
at com.facebook.testing.screenshot.internal.AlbumImpl.<init>(AlbumImpl.java:44)
at com.facebook.testing.screenshot.internal.AlbumImpl.create(AlbumImpl.java:49)
at com.facebook.testing.screenshot.internal.ScreenshotImpl.create(ScreenshotImpl.java:66)
at com.facebook.testing.screenshot.internal.ScreenshotImpl.getInstance(ScreenshotImpl.java:84)
at com.facebook.testing.screenshot.Screenshot.snap(Screenshot.java:38)
fespinoza commented 5 years ago

I still don’t manage to bootstrap the snapshot tests, I still get the same stack trace

java.lang.RuntimeException: Failed to create the directory /sdcard/screenshots/com.schibsted.peil.android.dev.test/screenshots-default for screenshots. Is your sdcard directory read-only?

I tried using the adb command to create the directory manually in the sdcard, then it fails with another error

java.lang.NullPointerException: Attempt to get length of null array
at com.facebook.testing.screenshot.internal.AlbumImpl.cleanup(AlbumImpl.java:131)
at com.facebook.testing.screenshot.internal.ScreenshotImpl.create(ScreenshotImpl.java:67)
at com.facebook.testing.screenshot.internal.ScreenshotImpl.getInstance(ScreenshotImpl.java:84)

I was reading the code of the library and seem just issues with creating the directories and cleaning the files, in com.facebook.testing.screenshot.internal.AlbumImpl.cleanup(AlbumImpl.java:131)

The following lines fail

for (String s : mDir.list()) {
  new File(mDir, s).delete();
}

Because mDir.list() returns null, which according to the [documentation](https://docs.oracle.com/javase/7/docs/api/java/io/File.html#list())

An array of strings naming the files and directories in the directory denoted by this abstract pathname. The array will be empty if the directory is empty. Returns null if this abstract pathname does not denote a directory, or if an I/O error occurs.

All of this seems quite weird to be honest. Have you tried to use screenshot-tests-for-android in a recent project?


I attach some extra code for completeness-sake My test manifest (I also added the write permissions to the app’s manifest)

```xml ``` ```kotlin package com.schibsted.peil import android.content.Intent import android.widget.LinearLayout import androidx.test.InstrumentationRegistry import com.facebook.testing.screenshot.Screenshot import com.facebook.testing.screenshot.Screenshot.snapActivity import com.facebook.testing.screenshot.ViewHelpers import com.schibsted.peil.feature.feed.FeedActivity import com.schibsted.peil.feature.feed.FooterStoryViewHolder import org.junit.Test class ThirteenCardScreenshotTest { @Test fun testScreenshot() { val context = InstrumentationRegistry.getInstrumentation().targetContext val view = LinearLayout(context) val viewHolder = FooterStoryViewHolder(view) view.addView(viewHolder.itemView) ViewHelpers.setupView(view) .setExactWidthDp(360) .setExactHeightDp(740) .layout() Screenshot.snap(view) .record() } } ```
lwasyl commented 5 years ago

Can we re-open this to indicate it's still a problem? I'm having the same issue and suggested workaround doesn't work

xiphirx commented 5 years ago

Sure, I'm not seeing this issue locally so I'm not sure how best to help...

andrzejchm commented 5 years ago

this happens on emulators with API level 28 only when trying to run the tests for the first time, to reproduce:

adb uninstall {APP_PACKAGE_HERE}
adb uninstall {TEST_PACKAGE_HERE}
adb shell
rm -rf /sdcard/screenshots
exit

and then simply run the test from android studio or using ./gradlew recordDebugAndroidTestScreenshotTest.

I also tried to add

    @get:Rule
    var permissionRule = GrantPermissionRule.grant(
            android.Manifest.permission.READ_EXTERNAL_STORAGE,
            android.Manifest.permission.WRITE_EXTERNAL_STORAGE
    )

to my tests, but that didn't solve the problem either

aahlenst commented 5 years ago

This problem happens on Android 10 (Q), too, due to the introduction of scoped external storage. Adding android:requestLegacyExternalStorage="true" to the <application> in the AndroidManifest.xml helps as a temporary workaround.

pedrovgs commented 4 years ago

Hi, @xiphirx this issue seems to be related to the relation between the APK generated with your test code and the APK generated with your production code. The first time you install both APKs while running the instrumentation tests from Android Studio or the command line an ID is assigned to both APKs. However, if for some reason this id is not the same for both APKs, the testing APK will not be able to access the production APK resources. This is Id can be configured in the application node declaration inside any AndroidManifest with this label: android:sharedUserId. The Andorid docs specify this about this param:

The name of a Linux user ID that will be shared with other apps. By default, Android assigns each app its own unique user ID. However, if this attribute is set to the same value for two or more apps, they will all share the same ID — provided that their certificate sets are identical. Apps with the same user ID can access each other's data and, if desired, run in the same process.

I don't know why the error seems to be related to the directory creation. The folder where the screenshots are being recorded is in the SD card and the permission is granted. Assigning different IDs seems to the testing APK and the production APK seems to be related but I don't fully understand why because I've got this library working in projects with the default sharedUserId configuration.

I found the solution because a shot library user reported me this error and we created a sample project where (I don't understand why) Android generates different id's for the testing and production APKs.

To fix this you can configure your testing AndoridManifest with the same id the production APK uses. You can use a special flavor or build type AndroidManiefst if you don't want to add this param to your production manifest

This could be the testing manifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.snapshottesting.test"
    android:sharedUserId="com.example.snapshottesting.uid">
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
</manifest>

and this the production manifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.snapshottesting"
    android:sharedUserId="com.example.snapshottesting.uid">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme"
        >
        <activity
            android:name=".MainActivity"
            android:label="@string/app_name"
            android:theme="@style/AppTheme.NoActionBar">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

I'm uploading here two Android projects where you can see the error. The first one will fail when running the tests and the second one, with the same code but adding the userSharedId config will work. You can run the tests by executing ./gradlew executeScreenshotTests -Precord.

SnapshotTesting failing.zip SnapshotTesting working.zip

I hope it helps 😃

TimoPtr commented 3 years ago

The issue is also present on the API 30 with the new scoped storage. I did try to fix it by using getExternalFileDirs(Environment.PICTURES) but I forgot that connectedAndroidTest removes both apk and it's scoped storage.... It seems that the only solution could be to grant MANAGE_EXTERNAL_STORAGE but I did not manage to do it properly with UiAutomation shell command

        String.format(Locale.ENGLISH, "appops set --uid %s MANAGE_EXTERNAL_STORAGE allow", context.getPackageName());

Calling this from the targetContext crash the process and on mContext does nothing.

If someone have an alternative idea I can try to implement it.

TimoPtr commented 3 years ago

I did find a hack to scoped storage I'm going to open a PR within the day

raafaelima commented 2 years ago

Hello Everyone, I'm having the same issues after updating my project to use API 30. Do you folks have any updates about this issue?

Caused by: java.lang.RuntimeException: Failed to create the directory /sdcard/screenshots/com.mayapp.test/screenshots-default for screenshots. Is your sdcard directory read-only?
    at com.facebook.testing.screenshot.internal.ScreenshotDirectories.getSdcardDir(ScreenshotDirectories.java:126)
    at com.facebook.testing.screenshot.internal.ScreenshotDirectories.get(ScreenshotDirectories.java:54)
bmattoso commented 2 years ago

Same issue here. Android emulator: Pixel 2, with Play Store. Android API: 30 (Android 11)

Screenshot tests were running fine on API 29 for me.

Did anyone discover a workaround for this?

ankur-rise commented 2 years ago

Same issue here on Android emulator 30 (Android 11)

hhaietk commented 2 years ago

Same issue on Android emulator API 28 without google API

I tried all following possible solutions:

If I tried to grant rule in runtime I get exception that this permission can not be granted..

@get:Rule
    var permissionRule = GrantPermissionRule.grant(
            android.Manifest.permission.READ_EXTERNAL_STORAGE,
            android.Manifest.permission.WRITE_EXTERNAL_STORAGE
    )
rodor87 commented 2 years ago

Same issue here on Android 11 API 30

mecoFarid commented 2 years ago

Hi, @xiphirx this issue seems to be related to the relation between the APK generated with your test code and the APK generated with your production code. The first time you install both APKs while running the instrumentation tests from Android Studio or the command line an ID is assigned to both APKs. However, if for some reason this id is not the same for both APKs, the testing APK will not be able to access the production APK resources. This is Id can be configured in the application node declaration inside any AndroidManifest with this label: android:sharedUserId. The Andorid docs specify this about this param:

The name of a Linux user ID that will be shared with other apps. By default, Android assigns each app its own unique user ID. However, if this attribute is set to the same value for two or more apps, they will all share the same ID — provided that their certificate sets are identical. Apps with the same user ID can access each other's data and, if desired, run in the same process.

I don't know why the error seems to be related to the directory creation. The folder where the screenshots are being recorded is in the SD card and the permission is granted. Assigning different IDs seems to the testing APK and the production APK seems to be related but I don't fully understand why because I've got this library working in projects with the default sharedUserId configuration.

I found the solution because a shot library user reported me this error and we created a sample project where (I don't understand why) Android generates different id's for the testing and production APKs.

To fix this you can configure your testing AndoridManifest with the same id the production APK uses. You can use a special flavor or build type AndroidManiefst if you don't want to add this param to your production manifest

This could be the testing manifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.snapshottesting.test"
    android:sharedUserId="com.example.snapshottesting.uid">
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
</manifest>

and this the production manifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.snapshottesting"
    android:sharedUserId="com.example.snapshottesting.uid">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme"
        >
        <activity
            android:name=".MainActivity"
            android:label="@string/app_name"
            android:theme="@style/AppTheme.NoActionBar">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

I'm uploading here two Android projects where you can see the error. The first one will fail when running the tests and the second one, with the same code but adding the userSharedId config will work. You can run the tests by executing ./gradlew executeScreenshotTests -Precord.

SnapshotTesting failing.zip SnapshotTesting working.zip

I hope it helps 😃

Didn't resolve anything.

This android:requestLegacyExternalStorage="true" works just fine tho

Venkat-juju commented 1 year ago

Same issue with API level 30.. Any solution.. seems like PR #273 also yet to be merged

Pirokar commented 1 year ago

Any solution? SharedUserId is deprecated, android:requestLegacyExternalStorage="true" is deprecated too