Xray Cloud Integration

[amitasil]

@medtoure18

Please review this PR once you get some time.

I am able to generate the required Access Token from the first request and passing it to the second API request but I am not able to change the Test Execution status in XRay. However, if I use the same request which I am sending through this code and paste it in Postman, I am able to change the status of Test Execution in XRay.

Body for the second request (working in Postman but not through the code): { "testExecutionKey": "SP-10","info" : {"startDate" : "2021-11-24T21:31:06+11:00", "finishDate" :"2021-11-24T21:31:06+11:00","revision": "001","description" : "Results of test execution ", "testEnvironments": ["Sandpit"]},"tests" : [{"testKey":"SP-8","status":"PASSED", "examples":[],"comment" : "Successful execution" }]}

Not sure exactly what am I missing. Maybe need to store/handle some cookies not sure.

FYI: you might need to increase the timeout value.

[medtoure18]

@amitasil there is an error in the json, it's PASS and not PASSED

{ "testExecutionKey": "SP-10","info" : {"startDate" : "2021-11-24T21:31:06+11:00", "finishDate" :"2021-11-24T21:31:06+11:00","revision": "001","description" : "Results of test execution ", "testEnvironments": ["Sandpit"]},"tests" : [{"testKey":"SP-8","status":"PASS", "examples":[],"comment" : "Successful execution" }]}

see xray doc

https://docs.getxray.app/display/XRAY/Import+Execution+Results+-+REST#ImportExecutionResultsREST-CucumberJSONresultsMultipart

[amitasil]

@medtoure18

This is the sample for xray cloud:

https://docs.getxray.app/display/XRAYCLOUD/Import+Execution+Results+-+REST+v2

[amitasil]

@medtoure18

It's too late here :)

Will do the changes and get back to you tomorrow.

Thanks!!

[medtoure18]

@amitasil Ok no problem, have a good night, its 2PM here :-)

[amitasil]

@medtoure18

Incorporated your review comments, but the issue still persists :(

[medtoure18]

What response body do you get with the second request ! ?

[amitasil]

@medtoure18

Just "object [Object]"

[medtoure18]

Try a console.log, it would shows you what's inside the objet

[amitasil]

@medtoure18

Made some updates and it's working fine now!!

This is what I am getting now:

SEND TO XRAY => { "testExecutionKey": "SP-10","info" : {"startDate" : "2021-11-25T11:10:10+11:00", "finishDate" :"2021-11-25T11:10:10+11:00","revision": "001","description" : "Results of test execution ", 
"testEnvironments": ["Sandpit"]},"tests" : [{"testKey":"SP-8","status":"PASSED", "examples":[],"comment" : "Successful execution" }]}
XRAY RESPONSE => eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0ZW5hbnQiOiJiMmMxZGFhZC0zOGViLTNiY2EtYTFmYy03OWIzODUwOTAwZDEiLCJhY2NvdW50SWQiOiI1NTcwNTg6MjlhMDgyN2YtODY5MC00YWM5LWIzZWUtNDE5MzA5Njg0ZGM4IiwiaXNYZWEiOmZhbHNlLCJpYXQiOjE2Mzc3OTkwMTMsImV4cCI6MTYzNzg4NTQxMywiYXVkIjoiOUI4MjA1QUE0OUMyNDc3MTgwMEM2ODg5NzZBQUZDQ0QiLCJpc3MiOiJjb20ueHBhbmRpdC5wbHVnaW5zLnhyYXkiLCJzdWIiOiI5QjgyMDVBQTQ5QzI0NzcxODAwQzY4ODk3NkFBRkNDRCJ9.wpDfvZ_N5vGBjf5PE9WTKUIYc98qcm52PmA6NHnzpCE
Access token generate successfully!
XRAY RESPONSE => {"id":"10018","key":"SP-10","self":"https://amitrawat.atlassian.net/rest/api/2/issue/10018"}
Test Execution updated successfully!

[medtoure18]

@amitasil i will create the new npm version this week-end thank you :-)

[amitasil]

Cheers @medtoure18 !! :)

[medtoure18]

@amitasil just publish the new npm version, can you check if it's ok for you. i removed request dependancy and updated xray_cloud config as default false.

[amitasil]

Thanks @medtoure18 !!

Will try this today.

[amitasil]

Hi @medtoure18 , the new changes are looking good.

One observation though. When I am running my two tests in parallel with 'testsExportedFromTestExecution' as false. I am getting two test runs getting generated in Xray Jira for the two executed tests. Each test run has one test case with the correct executed status.

Can we club all the tests executed in parallel and generate only one test run in Xray Jira, just like a sequential run?

This one could help: https://codecept.io/parallel/#parallel-execution-by-workers

[medtoure18]

@amitasil Thanks for the test :-) About parallèle workers, i think we need to change a lot of things. Because we need to know when the last parallèle test is finished before we send the results. Otherwise we could creat the test execution empty before all tests. If you have the time to do another PR you can :-)

[amitasil]

Hi @medtoure18 ,

I tried but could find a solution on how to handle Codeceptjs workers :(

We are planning to run our tests in parallel and upload results back in Xray and this issue has become a blocker for us...

Could you please look into this?

Thanks!!

[medtoure18]

Hi @amitasil, I dont have the time to work on the feature but i can check codeceptjs doc to find where can we find the solution, so it could help you for your PR

[amitasil]

Thanks @medtoure18 !!

[medtoure18]

Hi @amitasil,

I think the solution is to use the event: event.all.before for an empty test execution creation. after that we save the id of the test execution after each the same execution will be used.

right now we are using event.all.after to send the results to xray. see https://codecept.io/hooks/#event-listeners

[amitasil]

Hi @medtoure18

I am trying to use something like this the 'parallel.js' option they have mentioned at https://codecept.io/parallel/#parallel-execution:

But the thing is, I need to run the tests using "node parallel.js" command.

And I am not getting the test tags as an object, see the test details below:

{
   "type":"test",
   "title":"Navigate to Business Insurance page @TEST_SP-12 @TESTSET_SP-9 @feature",
   "body":"() => { }",
   "async":0,
   "sync":true,
   "_timeout":2000,
   "_slow":75,
   "_retries":-1,
   "timedOut":false,
   "_currentRetry":0,
   "pending":false,
   "id":"Gp2WtZ5crrM7g8XPpR2h+w",
   "workerIndex":1,
   "retries":-1,
   "duration":5586,
   "err":null,
   "parent":{
      "title":"Navigate to Business Insurance page @REQ_SP-11",
      "ctx":{

      },
      "suites":[

      ],
      "tests":[

      ],
      "root":false,
      "pending":false,
      "_retries":-1,
      "_beforeEach":[

      ],
      "_beforeAll":[

      ],
      "_afterEach":[

      ],
      "_afterAll":[

      ],
      "_timeout":2000,
      "_slow":75,
      "_bail":false,
      "_onlyTests":[

      ],
      "_onlySuites":[

      ],
      "delayed":false,
      "_events":{

      },
      "_eventsCount":1
   }
}

The above extract comes under this code:

workers.on(event.test.after, function (test) {
output.print(JSON.stringify(test));
});

Not sure how to get this thing working...