shouldProvideReportAfterCrashAsync

[moatazeldebsy]

Hi there, we are facing an issue when the app is crashing the allure not saved the results, and we found this TODO in the samples also, So any updates about this issue or how we can handle it ?

/**
     * TODO
     * When the app crashes in the background, the allure results are not saved.
     * That happens due to RunListener not invoking test finish/failure callbacks.
     * This seems like an Android Instrumentation bug but needs further investigation.
     */

Thanks

[kamildziadek]

Hey @moatazeldebsy. Android instrumentation process crashes in such case and we can't save the results to Android storage in such case. I have a couple of solutions in mind, but all of them are in POC phase. We might either change the way results are being saved or apply a workaround on Android instrumentation.

The best solution so far on my mind (without changing the internals of allure runner) is to register global Java error uncaught exception handler, report an exception to allure and flush the results to storage. I haven't tested this solution yet though. Unfortunately, I am a bite busy lately with my personal stuff and can't put much time into this project. Any solutions or contribution from your side is very welcome though :)

[matreshkin]

This runner is the implementation on the idea above. Attaches are ignored, still we have steps and exceptions in reports (there were no reports at all before).

// Based on https://github.com/allure-framework/allure-kotlin/blob/master/allure-kotlin-android/src/main/kotlin/io/qameta/allure/android/runners/AllureAndroidJUnitRunners.kt
// With UncaughtExceptionHandler feature.
@Suppress("unused")
class AllureAndroidRunner : AndroidJUnitRunner() {

    override fun onCreate(arguments: Bundle) {
        Allure.lifecycle = createDefaultAllureAndroidLifecycle()

        val listenerArg = listOfNotNull(
            arguments.getCharSequence("listener"),
            AllureJunit4Wrapper::class.java.name,
            ExternalStoragePermissionsListener::class.java.name.takeIf { useTestStorage }
        ).joinToString(separator = ",")
        arguments.putCharSequence("listener", listenerArg)

        super.onCreate(arguments)

        // To report crashes, see https://github.com/allure-framework/allure-kotlin/issues/32
        Thread.setDefaultUncaughtExceptionHandler { t, e -> // ! carefully override default UncaughtExceptionHandler
            AllureJunit4Wrapper.instance?.testCrashed(t, e)
        }
    }

    private fun createDefaultAllureAndroidLifecycle(): AllureAndroidLifecycle {
        if (useTestStorage) {
            return AllureAndroidLifecycle(TestStorageResultsWriter())
        }
        return AllureAndroidLifecycle()
    }

    private val useTestStorage: Boolean
        get() = PropertiesUtils.loadAllureProperties()
            .getProperty("allure.results.useTestStorage", "false")
            .toBoolean()

    class AllureJunit4Wrapper : RunListener() {

        val base = AllureJunit4()

        private var currentTestCase: String? = null

        init {
            instance = this
        }

        fun testCrashed(t: Thread, exception: Throwable) {
            currentTestCase?.let { uuid ->
                base.lifecycle.setCurrentTestCase(uuid)
                base.lifecycle.updateTestCase(uuid) { testResult: TestResult ->
                    with(testResult) {
                        status = ResultsUtils.getStatus(exception)
                        statusDetails = ResultsUtils.getStatusDetails(exception)
                    }
                }
                base.lifecycle.stopTestCase(uuid)
                base.lifecycle.writeTestCase(uuid)
            }
            currentTestCase = null
        }

        override fun testRunStarted(description: Description) = base.testRunStarted(description)

        override fun testRunFinished(result: Result) = base.testRunFinished(result)

        override fun testSuiteStarted(description: Description) = base.testSuiteStarted(description)

        override fun testSuiteFinished(description: Description) = base.testSuiteFinished(description)

        override fun testStarted(description: Description) {
            base.testStarted(description)
            currentTestCase = base.lifecycle.getCurrentTestCase()
        }

        override fun testFinished(description: Description) {
            base.testFinished(description)
            currentTestCase = null
        }

        override fun testFailure(failure: Failure) = base.testFailure(failure)

        override fun testAssumptionFailure(failure: Failure) = base.testAssumptionFailure(failure)

        override fun testIgnored(description: Description) = base.testIgnored(description)

        companion object {
            var instance: AllureJunit4Wrapper? = null
        }
    }
}