Unable to Retrieve Skipped/Pending Test Results via REST API after Execution

[Anveshd]

Describe the bug Dear Testkube Team,

I hope this message finds you well. I would like to bring to your attention a bug I encountered while working with Testkube.

Issue: Currently, there seems to be a limitation in retrieving skipped and pending test results via the REST API after test execution. When attempting to fetch the test case results through the API, only passed and failed results are being returned. This poses a challenge as it prevents users from accessing crucial information about skipped and pending test cases.

To Reproduce Steps to reproduce the behavior:

1. Executed a test containing skipped and pending test cases using Testkube.

2. config

   describe('Example to Demostrate the use of skip in cypress', () => {
   before(() => {
       cy.visit('https://wikipedia.org')
   })
   
   it.skip('Validate Page Title', () => {
       cy.title().should('eq', 'Wikipedia')
   })
   
   it('Search for Google Wiki Page', () => {
       cy.get('#searchInput').type('google')
       cy.get('button[type="submit"]').click()
   })
   
   it.skip('Validate Google Wiki Page has opened', () => {
       cy.get('h1#firstHeading').contains('Google')
       cy.title().should('eq', 'Google - Wikipedia')
   })
   })

3. Execution results in the testkube ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 3 │ │ Passing: 1 │ │ Failing: 0 │ │ Pending: 1 │ │ Skipped: 1 │ │ Screenshots: 0 │ │ Video: true │ │ Duration: 39 seconds │ │ Spec Ran: skipped.cy.js │ └────────────────────────────────────────
4. Execution results using restapi api: http://localhost:8088/v1/executions/{execution_id} results:

   "steps": [
     {
       "name": "Example to Demostrate the use of skip in cypress - Example to Demostrate the use of skip in cypress Search for Google Wiki Page",
       "duration": "28.704s",
       "status": "passed"
     }
   ]

   Observed that only passed and failed results are returned, while skipped and pending results are missing.

Expected behavior I kindly request that the Testkube platform be enhanced to include support for retrieving all test case results, including those that are marked as skipped or pending, through the REST API. This would greatly improve the usability and effectiveness of the platform, as it would provide a comprehensive overview of the test execution status.

Version / Cluster

• Which testkube version?
• What Kubernetes cluster? (e.g. GKE, EKS, Openshift etc, local KinD, local Minikube)
• What Kubernetes version?

Screenshots If applicable, add CLI commands/output to help explain your problem.

Additional context Add any other context about the problem here.

[vsukhin]

thank you @Anveshd for catching this. we will check out

[naveenreddymanukonda]

@vsukhin any update when this will be released

[vsukhin]

@Anveshd hopefully we can check it this week

[vsukhin]

Hi, @Anveshd Sounds like there is a confusion between Testkube Test and its Status and tests inside of Testkube Test and their statuses. Technically single Testkube Test can can contain hundreds/thousands of tests for particulat test engine that processes them, like in your k6 example. We don't even have skipped or pending statuses for Testkube Tests, these are statuses for k6 tests. For Testkube your scrript is just a signle test with Testkube Test

[Anveshd]

Hi @vsukhin We would like to inquire about the skipped or pending status for Cypress test results, not for k6 tests. We have observed that these results are visible in the Testkube (UI), but not in the API. Could you please confirm if our understanding is correct?

[vsukhin]

Yes, probably, you see them in test output logs. We take these values from JUnit report to pass it to our API, I can see skipped value as well, but I don't see pended one, but we can try both

[Anveshd]

Hi @vsukhin is above changes deployed in the testkube ? any ETA ?

[Anveshd]

Hi @vsukhin is above changes deployed in the testkube ? any ETA ?

[vsukhin]

I can deploy it today

[vsukhin]

But I have a question - what the name of the status to check - "pended" or "pending"? I could find only "skipped"

[Anveshd]

Pending only.

[Anveshd]

Thank you @vsukhin Could you please share me respective Helm chart.

[vsukhin]

Use latest one, like testkube-1.13.30