Android tests - Wrong test result when a RuntimeException is thrown

[alexandre-lefranc]

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

I'm submitting a ...

• [x] bug report
• [ ] feature request
• [ ] support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

When using Android tests and throwing a RuntimeException in the app, the test result is Success instead of Broken or Failed.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

class CrashActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        throw IllegalStateException("Crashing!")
   }
}

Please tell us about your environment:

Allure kotlin -> 2.4.0 Allure version (to generate reports) -> 2.27.0

Other information

I suggest to set a default test result status in fun testStarted()

    override fun testStarted(description: Description) {
        val uuid = testCases.get()
        val result = createTestResult(uuid, description).apply {
             status = Status.BROKEN
        }
        lifecycle.scheduleTestCase(result)
        lifecycle.startTestCase(uuid)
    }

[Yasmidrog]

Getting the same problem, but instead of Success results there are no results at all and no JSONs in the reports directory.

[MisterSawyer007]

Getting the same problem I think it is critical, since you can skip the crash and not notice that the test has not been added to the report no results if there was a bug in the developer's code: which caused the crash (reproduced with: NullPointerExeception, IllegalStateException etc)