CucumberJUnitRunnerBuilder throws CucumberException when finishing report (Stream closed)

👓 What did you see?

Dear Team, I try to integrate the cucumber-android library into my Android (Kotlin) project. I can run it, execute tests and even get the reports stored on the device but every execution of ends up with the following failure:

gary0707@xyz123:~/git/myApp$ adb shell am instrument -w -e tags @login com.gary0707.app.test/.AppMockedUiTestAndroidJUnitRunner

Manage tab allows the use to manage Digital Content on user device:.

Time: 5.781
There was 1 failure:
1) io.cucumber.junit.CucumberJUnitRunnerBuilder
cucumber.runtime.CucumberException: Error while transforming.
        at cucumber.runtime.formatter.JUnitFormatter.finishReport(JUnitFormatter.java:161)
        at cucumber.runtime.formatter.JUnitFormatter.access$400(JUnitFormatter.java:41)
        at cucumber.runtime.formatter.JUnitFormatter$5.receive(JUnitFormatter.java:81)
    at cucumber.runtime.formatter.JUnitFormatter$5.receive(JUnitFormatter.java:78)
    at cucumber.runner.AbstractEventPublisher.send(AbstractEventPublisher.java:45)
    at cucumber.runner.AbstractEventBus.send(AbstractEventBus.java:9)
    at cucumber.runner.TimeServiceEventBus.send(TimeServiceEventBus.java:3)
    at io.cucumber.junit.CucumberJUnitRunner$1.evaluate(CucumberJUnitRunner.java:287)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:27)
    at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
    at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
    at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:444)
    at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2205)
Caused by: javax.xml.transform.TransformerException: java.io.IOException: Stream closed
    at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:401)
    at cucumber.runtime.formatter.JUnitFormatter.finishReport(JUnitFormatter.java:158)
    ... 23 more
Caused by: java.io.IOException: Stream closed
    at org.apache.xml.serializer.ToXMLStream.startDocumentInternal(ToXMLStream.java:169)
    at org.apache.xml.serializer.SerializerBase.startDocument(SerializerBase.java:1190)
    at org.apache.xalan.transformer.TransformerIdentityImpl.flushStartDoc(TransformerIdentityImpl.java:941)
    at org.apache.xalan.transformer.TransformerIdentityImpl.startElement(TransformerIdentityImpl.java:1072)
    at org.apache.xml.serializer.TreeWalker.startNode(TreeWalker.java:359)
    at org.apache.xml.serializer.TreeWalker.traverse(TreeWalker.java:145)
    at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:390)
    ... 24 more

FAILURES!!!
Tests run: 1,  Failures: 1

All the instrumentation code is included into my app module: app/src/androidTest. Gherkin scenarios are located in the androidTest/assets/features folder while steps under the usual androidTest/kotlin/src/com.gary0707.... folders.

I am using the derived extension of the CucumberAndroidJUnitRunner. Mostly what you provide in the sample Cukeculator app (but I have migrated it to Kotlin):

package com.gary0707.app.test

import android.app.Application
import android.content.Context
import android.os.Bundle
import io.cucumber.android.runner.CucumberAndroidJUnitRunner
import io.cucumber.junit.CucumberOptions
import java.io.File

/**
 * The CucumberOptions annotation is mandatory for exactly one of the classes in the test project.
 * Only the first annotated class that is found will be used, others are ignored. If no class is
 * annotated, an exception is thrown. This annotation does not have to placed in runner class
 */
@CucumberOptions(
    glue = ["com.gary0707.app.test.bdd.cucumber.steps"],
    tags = ["not @ignore"],
    features = ["features"],
    strict = true,
)
class ChartManagerMockedUiTestAndroidJUnitRunner : CucumberAndroidJUnitRunner() {
    override fun newApplication(cl: ClassLoader?, className: String?, context: Context?): Application {
        return super.newApplication(cl, KoinTestApp::class.java.name, context)
    }

    override fun onCreate(bundle: Bundle) {
        bundle.putString("plugin", getPluginConfigurationString()) // we programmatically create the plugin configuration
        File(getAbsoluteFilesPath()).mkdirs()
        super.onCreate(bundle)
    }

    /**
     * Since we want to checkout the external storage directory programmatically, we create the plugin configuration
     * here, instead of the [CucumberOptions] annotation.
     *
     * @return the plugin string for the configuration, which contains XML, HTML and JSON paths
     */
    private fun getPluginConfigurationString(): String? {
        val cucumber = "cucumber"
        val separator = "--"
        return "junit:" + getCucumberXml(cucumber) + separator +
            "html:" + getCucumberHtml(cucumber)
    }

    private fun getCucumberHtml(cucumber: String): String {
        return getAbsoluteFilesPath() + "/" + cucumber + ".html"
    }

    private fun getCucumberXml(cucumber: String): String {
        return getAbsoluteFilesPath() + "/" + cucumber + ".xml"
    }

    /**
     * The path which is used for the report files.
     *
     * @return the absolute path for the report files
     */
    private fun getAbsoluteFilesPath(): String {

        //sdcard/Android/data/cucumber.cukeulator
        val directory: File? = targetContext.getExternalFilesDir(null)
        return File(directory, "reports").absolutePath
    }
}

✅ What did you expect to see?

No failure caused by lib internals. Tests should fail if scenarios fails

📦 Which tool/library version are you using?

cucumberAndroid : '4.8.1' cucumberPicoContainer: '4.8.1' kotlin : '1.7.22'

Running on Android 11 devices.

🔬 How could we reproduce it?

The sample Cukeculator project, cloned from the repo works fine. Once migrated to Kotlin then the problem occurs.

I have debugged the library during the test execution and indeed the: cucumber.runtime.formatter.JUnitFormatter.finishReport(JUnitFormatter.java) becomes closed while it's still going to be used. I didn't manage to identify the root cause of that: CucumberJUnitRunner.java#L293

Steps to reproduce the behavior:

Install the sample app
Install its instrumentation app
execute instrumentation via ADB but using also -r option to see the output on the terminal
See error I have copy&paste on top of this post.

📚 Any additional context?

YES: test reports on the device always contain duplicated result. If I execute a single scenario, the report looks like the scenario was executed twice When I execute the adb shell am instrument ... with option -r I see duplicated entries from the TestRunner, despite the fact I have only on Feature with One scenario:

adb shell am instrument -r -w com.gary0707.app.test/.ChartManagerMockedUiTestAndroidJUnitRunner
INSTRUMENTATION_STATUS: class=Empty frag allows nothing special
INSTRUMENTATION_STATUS: current=1
INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
INSTRUMENTATION_STATUS: numtests=1
INSTRUMENTATION_STATUS: stream=
Empty frag allows nothing special:
INSTRUMENTATION_STATUS: test=Clicking on button does nothing
INSTRUMENTATION_STATUS_CODE: 1
INSTRUMENTATION_STATUS: class=Empty frag allows nothing special
INSTRUMENTATION_STATUS: current=1
INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
INSTRUMENTATION_STATUS: numtests=1
INSTRUMENTATION_STATUS: stream=.
INSTRUMENTATION_STATUS: test=Clicking on button does nothing
INSTRUMENTATION_STATUS_CODE: 0
INSTRUMENTATION_RESULT: stream=

I have a feeling that something is executed twice. Like two hanging (due to leak?) activities.

This text was originally generated from a template, then edited by hand. You can modify the template here.

cucumber / cucumber-android