oppia / oppia-android

A free, online & offline learning platform to make quality education accessible for all.
https://www.oppia.org
Apache License 2.0
300 stars 502 forks source link

Fix NPS Survey Gating #5356

Closed adhiamboperes closed 2 months ago

adhiamboperes commented 3 months ago

Explanation

I found out during testing that there is an underlying gating DataProvider update behavior that occurs because of the way I initially designed the survey popup behaviour:

I mitigated this by removing the observer once the initial gating result has been received and processed.

Tests

I added tests that test survey gating based on changes in the time criterion, while other gating criteria remains constant/fulfilled. Exhaustive gating tests were added in the SurveyGatingController.

I also made a change in the StateFragmentTest that unregisters the idling resource after initializing profiles instead of at the end of the test runs. Previously, StateFragmentTestwould not run locally and fail with the error:

Error performing 'single click - At Coordinates: 173, 329 and precision: 16, 16' on view 'with id: org.oppia.android:id/play_test_exploration_button'.
at androidx.test.espresso.PerformException$Builder.build(PerformException.java:86)
at androidx.test.espresso.base.DefaultFailureHandler.getUserFriendlyError(DefaultFailureHandler.java:87)
at androidx.test.espresso.base.DefaultFailureHandler.handle(DefaultFailureHandler.java:59)
at androidx.test.espresso.ViewInteraction.waitForAndHandleInteractionResults(ViewInteraction.java:322)
at androidx.test.espresso.ViewInteraction.desugaredPerform(ViewInteraction.java:178)
at androidx.test.espresso.ViewInteraction.perform(ViewInteraction.java:119)
at org.oppia.android.app.player.state.StateFragmentTest.startPlayingExploration(StateFragmentTest.kt:475)org.oppia.android.app.player.state.StateFragmentTest.testFinishChapter_lateNight_isPastGracePeriod_minimumAggregateTimeMet_noSurveyPopup(StateFragmentTest.kt:264)
Caused by: androidx.test.espresso.IdlingResourceTimeoutException: Wait for [TestCoroutineDispatcherIdlingResource] to become idle timed out
at androidx.test.espresso.IdlingPolicy.handleTimeout(IdlingPolicy.java:61)
at androidx.test.espresso.base.UiControllerImpl$5.resourcesHaveTimedOut(UiControllerImpl.java:434)
at androidx.test.espresso.base.IdlingResourceRegistry$Dispatcher.handleTimeout(IdlingResourceRegistry.java:481)

This was caused by registering an idling resource before test begins, then when the activity starts, the main thread becomes busy. Generally, the test will continue when the main thread becomes idle, but the idling resource now blocks it and does not call onTransitionToIdle(). So it will always time out. Unregistering the idling resource before the activity starts frees up the main thread.

Disabled running on Robolectric due to known issue with Drag and Drop interaction.

Espresso Run Screenshot Screenshot 2024-03-13 at 19 35 12

Essential Checklist

For UI-specific PRs only

If your PR includes UI-related changes, then:

adhiamboperes commented 3 months ago

@BenHenning, PTAL.

BenHenning commented 3 months ago

@adhiamboperes is it possible to add tests for any of the discovered user-facing problems?

adhiamboperes commented 3 months ago

@adhiamboperes is it possible to add tests for any of the discovered user-facing problems?

@BenHenning, I have added some UI tests and updated the PR description.

adhiamboperes commented 3 months ago

@BenHenning, PTAL.

oppiabot[bot] commented 2 months ago

Hi @adhiamboperes, this PR is ready to be merged. Please address any remaining comments prior to merging, and feel free to merge this PR once the CI checks pass and you're happy with it. Thanks!

adhiamboperes commented 2 months ago

Resolved all comments as fixed and enabling auto merge.