Xray-App / xray-junit-extensions

Enhanced JUnit 5 integration with Xray Test Management for Jira
Eclipse Public License 2.0
16 stars 8 forks source link

add timestamp to report filename is not working as expected #21

Closed Alje001 closed 2 years ago

Alje001 commented 2 years ago

I have a issue using the following config value add_timestamp_to_report_filename When I put add_timestamp_to_report_filename=true, I'm not getting the xml file anymore. would you please support?

bitcoder commented 2 years ago

Hi, in order to be able to help you out, you need to provide more information, including the version of the plugin being used. The best would be to make a small/simple proof-of-concept code and share it, for example in a temporary github repo.

Alje001 commented 2 years ago

@bitcoder thank you for your response

I am using gradle with junit and I prepared a demo to show you the error:

build.gradle

`plugins { id 'java' id 'org.gradle.test-retry' version '1.4.0' }

group 'demo.test' version '1.0-SNAPSHOT'

wrapper() { gradleVersion = '7.5.1' distributionUrl = "https://services.gradle.org/distributions/gradle-${gradleVersion}-bin.zip" }

repositories { mavenCentral() }

dependencies { testImplementation 'org.junit.jupiter:junit-jupiter-api:5.9.0' testImplementation 'app.getxray:xray-junit-extensions:0.6.1' } test { dependsOn.cleanTestresult ignoreFailures = true retry { failOnPassedAfterRetry = false maxFailures = 10 maxRetries = 1 } useJUnitPlatform() //maxParallelForks = Runtime.runtime.availableProcessors().intdiv(2) ?: 1 }

task cleanTestresult { delete "${rootDir}/target/xray/junit-results/" }`

Junit Tests

`package demo;

import app.getxray.xray.junit.customjunitxml.XrayTestReporterParameterResolver; import app.getxray.xray.junit.customjunitxml.annotations.Requirement; import org.junit.Assert; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestInstance; import org.junit.jupiter.api.extension.ExtendWith;

@TestInstance(TestInstance.Lifecycle.PER_CLASS) @ExtendWith(XrayTestReporterParameterResolver.class) public class DemoTest {

@Test
@Requirement({ "ID-45" })
public void firstTest(){
    Assert.assertTrue(true);
}

@Test
@Requirement({ "ID-45" })
public void secondTest(){
    Assert.assertTrue(true);
}

}`

Under this path \resources\META-INF\services I created theFile org.junit.platform.launcher.TestExecutionListener with this value app.getxray.xray.junit.customjunitxml.EnhancedLegacyXmlReportGeneratingListener

I am also using xray-junit-extensions.properties to define result name and target path. report_directory=target/xray/junit-results report_filename=JunitXrayReport

When I put add_timestamp_to_report_filename=true I am getting this error :

Sep. 13, 2022 4:50:10 NACHM. org.junit.platform.launcher.core.CompositeTestExecutionListener lambda$notifyEach$19 WARNING: TestExecutionListener [app.getxray.xray.junit.customjunitxml.EnhancedLegacyXmlReportGeneratingListener] threw exception for method: executionFinished(TestIdentifier [uniqueId = [engine:junit-jupiter], parentId = null, displayName = 'JUnit Jupiter', legacyReportingName = 'JUnit Jupiter', source = null, tags = [], type = CONTAINER], TestExecutionResult [status = SUCCESSFUL, throwable = null]) java.nio.file.InvalidPathException: Illegal char <:> at index 29: JunitXrayReport-2022_09_13_16:50:10_098.xml

The path was not valid and can't be created.

My goal is to use parallel execution of junit test using the following code maxParallelForks = Runtime.runtime.availableProcessors().intdiv(2) ?: 1

But it looks like that the result was always overwritten. Therefore I was hopping that using add_timestamp_to_report_filename=true would solve the issue.

Would you please support me resolving this issue.

bitcoder commented 2 years ago

I don't know how maxParallelForks works and affects the Junit xml report generation. Can you try running and adapting this code to see if we figure out the cause? https://github.com/Xray-App/tutorials-java-gradle/tree/main/junit5_enhanced

note: the previous code is referred on the Gradle tutorial on Xray doc site: https://docs.getxray.app/display/XRAY/Integration+with+Gradle#IntegrationwithGradle-JUnit5withadditionalinformation

bitcoder commented 2 years ago

I've picked the example I mentioned, and changed the build.gradle to

test {
    useJUnitPlatform()
    maxParallelForks = Runtime.runtime.availableProcessors() //.intdiv(2) ?: 1
    reports {
        // destination = file('build/test-results/folder')
        junitXml.required = true
        html.required = false
    }
    ignoreFailures = true
}

I've also changed `src/test/resources/xray-junit-extensions.properties` to:

```bash
report_filename=JunitXrayReport
report_directory=reports
add_timestamp_to_report_filename=true

Then I ran gradle clean test and a file was generated with all the 5 test results:

$ ls reports/
JunitXrayReport-2022_09_13_16:56:19_587.xml

$ fgrep -c "<testcase" reports/JunitXrayReport-2022_09_13_16\:56\:19_587.xml
5

So, I'm unable to replicate your problem. Can you please try to replicate it on top of that code for example?

Alje001 commented 2 years ago

@bitcoder This is great. I think that the issue is the dateformat see my error: Sep. 13, 2022 6:43:01 NACHM. org.junit.platform.launcher.core.CompositeTestExecutionListener lambda$notifyEach$19 WARNING: TestExecutionListener [app.getxray.xray.junit.customjunitxml.EnhancedLegacyXmlReportGeneratingListener] threw exception for method: executionFinished(TestIdentifier [uniqueId = [engine:junit-jupiter], parentId = null, displayName = 'JUnit Jupiter', legacyReportingName = 'JUnit Jupiter', source = null, tags = [], type = CONTAINER], TestExecutionResult [status = SUCCESSFUL, throwable = null]) java.nio.file.InvalidPathException: Illegal char <:> at index 29: JunitXrayReport-2022_09_13_18:43:01_127.xml

my os doesn't like this file name JunitXrayReport-2022_09_13_18:43:01_127.xml. How can I change the dateformat?

I think that if date is 2022_09_13_18_43_01_127 I wouldn't face this issue.

bitcoder commented 2 years ago

Currently you can't. Which OS and OS version do you have exactly?

Alje001 commented 2 years ago

I am using Windows 10 and windows doesn't like this format because of this char ':'

Alje001 commented 2 years ago

Thank you for the support:)