Inconsistent screenshot-testing with paparazzi and GlideImage composable

[hhpettersen]

Description
I am encountering inconsistent image loading results in screenshot tests using Paparazzi and Glide. (At this point I am uncertain if the issue lies with Paparazzi or Glide). Sometimes the image from a URL loads correctly, but at other times the image appears blank, and occasionally, only the loading placeholder is displayed. This variability in results makes reliable UI testing challenging.

Paparazzi-version: 1.3.3 Glide-version: 1.0.0-beta01

Steps to Reproduce

1. Set up a screenshot test using Paparazzi with a GlideImage composable to load images.
2. Record screenshots.
3. Verify tests.
4. Observe inconsistent results across multiple test runs.

Composable Code:

val adImageSize = 50.dp
val requestSize = with(LocalDensity.current) { adImageSize.toPx().toInt() }

Box {
    GlideImage(
        model = ad?.image.orEmpty(),
        contentDescription = stringResource(R.string.content_description_adimage),
        contentScale = ContentScale.FillBounds,
        loading = placeholder(painterResource(id = R.drawable.placeholder)),
        failure = placeholder(painterResource(id = R.drawable.ad_placeholder)),
        modifier = Modifier
            .size(adImageSize)
            .align(Alignment.Center)
            .clip(shapes.roundedCornerExtraSmall),
        requestBuilderTransform = {
            it.apply(RequestOptions().override(requestSize))
        },
    )
}

Test Setup:

@RunWith(TestParameterInjector::class)
class ChatScreenshotTest(
    @TestParameter
    nightMode: NightMode,
) {
    @get:Rule
    val paparazzi = Paparazzi(
        deviceConfig = DeviceConfig.PIXEL_6_PRO.copy(
            nightMode = nightMode,
        ),
        theme = "android:Theme.Material.ActionBar",
        snapshotHandler = if (PaparazziConfig.isVerifying) {
            SnapshotVerifier(
                maxPercentDifference = PaparazziConfig.maxPercentDifference,
                rootDirectory = PaparazziConfig.dir
            )
        } else {
            HtmlReportWriter(snapshotRootDirectory = PaparazziConfig.dir)
        }
    )

    @Test
    fun textMessages() {
        paparazzi.snapshot {
            CompositionLocalProvider(
                LocalInspectionMode provides true,
            ) {
                Theme {
                    ConversationPreview(createPreviewTextMessages())
                }
            }
        }
    }
}

Expected behavior
The expected behavior is for the images to load consistently in each test case, either displaying the correct image, the placeholder, or the failure state without variance across test runs.

Additional information:

• Paparazzi Version: 1.3.3
• OS: macOS 14.2.1
• Compile SDK: 34
• Gradle Version: 8.3
• Android Gradle Plugin Version: 8.2.2

I also raised an issue on Glide since I am uncertain where the bug recides at the moment: https://github.com/bumptech/glide/issues/5402

[jrodbx]

Sometimes the image from a URL loads correctly, but at other times the image appears blank, and occasionally, only the loading placeholder is displayed.

It sounds like a race condition, as the network is involved in your test setup. I'm not familiar with how Glide handles its networking stack, but for Picasso, we encourage faking the image request routing to ensure consistent test results. See here for an example: https://github.com/square/picasso/blob/master/picasso-paparazzi-sample/src/test/java/com/example/picasso/paparazzi/PicassoPaparazziTest.kt