apter-tech / junit5-robolectric-extension

This repository aims to bridge the gap between JUnit 5 and Robolectric, enabling developers to leverage the benefits of both frameworks for unit testing Android applications. While Robolectric currently lacks a dedicated JUnit 5 extension, this project proposes a community-driven solution to achieve seamless integration.
Apache License 2.0
52 stars 3 forks source link

Espresso interactions deadlock somewhere after the first successful test #78

Open mannodermaus opened 4 months ago

mannodermaus commented 4 months ago

I have yet to dive deep into what causes this, but here is a side effect I have observed with Espresso assertions running in a JUnit 5 context that uses RobolectricExtension. It's probably the magic of switching thread contexts and class loaders that trips it up, but I wanted to hear your thoughts on the matter first. If I remove the Espresso stuff and only keep ActivityScenario around, then it seems to work no matter how many tests I run. Also, this happens with any kind of test (@ParameterizedTest, @RepeatedTest etc), but I am using the base @Test to make it as simple as possible.

With JUnit 5

The following test class will execute "test 1" successfully, then deadlock in "test 2" just before the first onView().check() assertion:

// ...
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith

@ExtendWith(RobolectricExtension::class)
class ActivityScenarioTest {
    @Test
    fun `test 1`() {
        println("1")
        val scenario = ActivityScenario.launch(MainActivity::class.java)

        println(" - check")
        onView(withText("Changed")).check(doesNotExist())
        println(" - click")
        onView(withText("Click me")).perform(click())
        println(" - verify")
        onView(withText("Changed")).check(matches(isDisplayed()))

        scenario.close()
    }

    @Test
    fun `test 2`() {
        println("2")
        val scenario = ActivityScenario.launch(MainActivity::class.java)

        println(" - check")
        onView(withText("Changed")).check(doesNotExist())
        println(" - click")
        onView(withText("Click me")).perform(click())
        println(" - verify")
        onView(withText("Changed")).check(matches(isDisplayed()))

        scenario.close()
    }
}
Screenshot 2024-07-22 at 15 47 24

With JUnit 4

When changing this to a JUnit 4 environment (i.e. change @Test annotations, then replace @ExtendWith with @RunWith), all tests are okay:

// ...
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner

@RunWith(RobolectricTestRunner::class)
class ActivityScenarioTest {
    @Test
    fun `test 1`() {
        println("1")
        val scenario = ActivityScenario.launch(MainActivity::class.java)

        println(" - check")
        onView(withText("Changed")).check(doesNotExist())
        println(" - click")
        onView(withText("Click me")).perform(click())
        println(" - verify")
        onView(withText("Changed")).check(matches(isDisplayed()))

        scenario.close()
    }

    @Test
    fun `test 2`() {
        println("2")
        val scenario = ActivityScenario.launch(MainActivity::class.java)

        println(" - check")
        onView(withText("Changed")).check(doesNotExist())
        println(" - click")
        onView(withText("Click me")).perform(click())
        println(" - verify")
        onView(withText("Changed")).check(matches(isDisplayed()))

        scenario.close()
    }
}
Screenshot 2024-07-22 at 15 50 00

I have pushed an example to a branch in my fork: https://github.com/mannodermaus/junit5-robolectric-extension/blob/check/androidx-robolectric-espresso-deadlock/integration-tests/agp-kotlin-dsl/src/test/kotlin/tech/apter/junit/jupiter/robolectric/integration/tests/agp/kotlin/dsl/ActivityScenarioTest.kt

mannodermaus commented 4 months ago

It took a bit of time, but I was finally able to trace this down to the fact that RobolectricExtension will reset the main looper after each test. This doesn't jive with Espresso's usage of Dagger to inject the original main looper as a singleton into all of its view assertions (link; search for provideMainLooper and provideMainThreadExecutor). After the first Robo+JUnit5 test destroys the main looper, the next test receives a different one and the chain is broken. Espresso will wait indefinitely on a result using a LinkedBlockingQueue in its InteractionResultsHandler (link), causing the blockage.

I'd like to inquire first about the reason to reset sMainLooper and sThreadLocal after a test. If I comment this out, my tests run, but obviously there was a reason you added this in the first place. Maybe we can find an alternative that keeps one main looper alive, but still clears stuff inside it (maybe through delegation)? Thanks in advance! Looking forward to the discussion.

warnyul commented 3 months ago

@mannodermaus The primary reason for resetting the loopers at the end of each test is to ensure that the looper mode instrumentation functions correctly. If the loopers are not reset, tests may run on the main thread rather than the intended instrumented thread. This behavior is managed by Robolectric's resetLoopers method in ShadowPausedLooper class which calls createMainThreadAndLooperIfNotAlive to set up the necessary threads for instrumentation testing. I am not sure why this behavior occurs with JUnit 5, considering it works fine with JUnit 4. I would welcome any further insights you might have.