Qase integration ver `2.1.3` for java11+maven+rest assured+junit5 project not sending test results to Qase dashobard.

TemirlanUtegenov commented 2 years ago

Hi, Qase team!

I tried to use Qase integration with my automation test project. My project: java11+maven+rest assured+junit5 According to https://github.com/qase-tms/qase-java/tree/master/qase-junit5 I made these steps:

add dependencies to maven project (attached below)
add CaseId to test (in my case @CaseId(1) , example attached below)
create test run with test in the qase dashboard (QASE_RUN_ID = 2)
create api token in the qase dashboard
In the project - run maven command with parameters. In my case it is mvn clean test -DQASE_ENABLE=true -DQASE_PROJECT_CODE=MYCHANCES -DQASE_RUN_ID=2 -DQASE_API_TOKEN=token

Expected result: There is test case execution result in my Test run on the Qase dashboard

Observed result: There is no test case execution result in my test run on the Qase dashboard

Am I doing something wrong? Could you try to reproduce the same problem on your side?

My annotations for test

    @Test
    @CaseId(1)
    @DisplayName("New User registration")
    public void newUserRegistrationTest() {
                       some code here}

My dependencies:

<dependencies>
    <dependency>
        <groupId>io.rest-assured</groupId>
        <artifactId>rest-assured</artifactId>
        <version>5.0.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.json</groupId>
        <artifactId>json</artifactId>
        <version>20220320</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.13.2.2</version>
    </dependency>
    <dependency>
        <groupId>net.andreinc</groupId>
        <artifactId>mockneat</artifactId>
        <version>0.4.8</version>
    </dependency>
    <dependency>
        <groupId>io.qase</groupId>
        <artifactId>qase-api</artifactId>
        <version>2.1.3</version>
    </dependency>
    <dependency>
        <groupId>io.qase</groupId>
        <artifactId>qase-java</artifactId>
        <version>2.1.3</version>
        <type>pom</type>
    </dependency>
    <dependency>
        <groupId>io.qase</groupId>
        <artifactId>qase-junit5</artifactId>
        <version>2.1.3</version>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M4</version>
            <configuration>
                <properties>
                    <configurationParameters>
                        junit.jupiter.extensions.autodetection.enabled = true
                    </configurationParameters>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>

savkk commented 2 years ago

Hi! Unfortunately, I cannot reproduce this problem on my side.

Can you add the following code to your test for debugging?

    @Test
    @CaseId(1)
    @DisplayName("New User registration")
    public void newUserRegistrationTest() {
        QaseClient.getApiClient().setDebugging(true);
    }

This code allows you to see what the client is sending to Qase TMS.

TemirlanUtegenov commented 2 years ago

Got it. Here are logs from my test launch.

Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: --> POST https://api.qase.io/v1/result/MYCHANCES/2/bulk
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: Content-Length: 542
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: Accept: application/json
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: Token:  **token**
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: Content-Type: application/json
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: X-CLIENT-REPORTER: JUnit 5
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: User-Agent: Qase API Client
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO:
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: {"results":[{"status":"passed","time_ms":2,"defect":false},{"status":"passed","time_ms":1,"defect":false},{"status":"passed","time_ms":0,"defect":false},{"status":"passed","time_
ms":0,"defect":false},{"status":"passed","time_ms":0,"defect":false},{"status":"passed","time_ms":0,"defect":false},{"status":"passed","time_ms":0,"defect":false},{"status":"passed","t
ime_ms":0,"defect":false},{"status":"passed","time_ms":1,"defect":false},{"case_id":1,"status":"passed","time_ms":2,"defect":false},{"status":"passed","time_ms":0,"defect":false}]}    
Apr 12, 2022 11:52:19 PM okhttp3.internal.platform.Platform log
INFO: --> END POST (542-byte body)
Apr 12, 2022 11:52:20 PM okhttp3.internal.platform.Platform log
INFO: <-- HTTP FAILED: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
th to requested target
[INFO] 
[INFO] Results:
[INFO]
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  16.980 s
[INFO] Finished at: 2022-04-12T23:52:21+03:00
[INFO] ------------------------------------------------------------------------

TemirlanUtegenov commented 2 years ago

okay, I fixed PKIX problem. there is a new log for my test run

HTTP/1.1 200
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 5.162 s - in api.tests.auth.RegistrationTest
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: --> POST https://api.qase.io/v1/result/MYCHANCES/2/bulk
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: Content-Length: 72
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: Accept: application/json
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: Token: token
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: Content-Type: application/json
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: X-CLIENT-REPORTER: JUnit 5
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: User-Agent: Qase API Client
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO:
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: {"results":[{"case_id":1,"status":"passed","time_ms":5,"defect":false}]}
Apr 13, 2022 12:27:02 AM okhttp3.internal.platform.Platform log
INFO: --> END POST (72-byte body)
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: <-- 403 Forbidden https://api.qase.io/v1/result/MYCHANCES/2/bulk (1510ms)
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: Date: Tue, 12 Apr 2022 21:27:04 GMT
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: Content-Type: application/json
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: Transfer-Encoding: chunked
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: Connection: keep-alive
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: Cache-Control: no-cache, private
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: Strict-Transport-Security: max-age=15724800; includeSubDomains
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO:
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: {"status":false,"errorMessage":"Bulk results submission is disabled for this project."}
Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log
INFO: <-- END HTTP (87-byte body)
[INFO] 
[INFO] Results:
[INFO]
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 10
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  16.888 s
[INFO] Finished at: 2022-04-13T00:27:05+03:00
[INFO] ------------------------------------------------------------------------

Interesting row: Apr 13, 2022 12:27:04 AM okhttp3.internal.platform.Platform log INFO: {"status":false,"errorMessage":"Bulk results submission is disabled for this project."}

What should I do with that?

n3r commented 2 years ago

Hello @TemirlanUtegenov!

You need to enable this option in project settings:

TemirlanUtegenov commented 2 years ago

Hello @TemirlanUtegenov!

You need to enable this option in project settings:

That's correct! now everything is working as expected. Was it mentioned in the any doc or instruction?

QaseClient.getApiClient().setDebugging(true); was very helpful. Will use it if get any trouble

qase-tms / qase-java

Qase integration ver `2.1.3` for java11+maven+rest assured+junit5 project not sending test results to Qase dashobard. #46