search

systelab / allure-reporter

Printable report generation for Allure
5 stars 2 forks source link

Upload Result to Jama fails when working with a big number of .json #43

Closed SCoboG closed 5 years ago

SCoboG commented 5 years ago

Describe the bug When it has loaded in the Allure Reporter a large number of .json files and the user select to upload the results to Jama, not all the results are uploaded, just a few ones.

To Reproduce Steps to reproduce the behavior:

  1. Have created in Jama a Test plan with test groups and a test cycle.
  2. Load in Allure Report a large number of .json files (at least 150)
  3. Upload the result to Jama selecting the correct Test plan, test cycle and test groups.
  4. See error

Expected behavior Only a few number of test cases are modified with the results of the executions.

olgapuig commented 5 years ago

After doing the investigation, the issue is that the call GET /testruns?testCycle=508307 is limited to 20 results. According to that, when a Test cycle contains more than 20 Test Runs, the other one's are not updated.

image image

ManelMoreno commented 5 years ago

Good news to have it identified! I can provide some info that could be useful for its resolution. I have learnt from Jordi Treserres (that is using JAMA REST API for querying Test Runs, TCs, Defects, etc..to build Test metrics in Excel), that this parameter can be changed to 50. But this is the maximum limit (at least for a GET action). If you want to extend for a bigger number you need to implement a way to iterate this GET (ask JT how)

ManelMoreno commented 5 years ago

He has also detected a performance issue with large amount of items, as the REST API seems to protect the server and it takes some seconds into answering with all the items requested.

ManelMoreno commented 5 years ago

Should we implement also a way to check that the same amount of Test Runs uploaded have been assigned with a status? or that all of them have the Test Status we wanted to upload? I mean, some easy mechanism to be sure (or at least provide detailed feedback) about the upload of the results

SCoboG commented 5 years ago

Retest scenario with result Pass. After applying the fix, it is possible to update the status of 57 test cases in the same test cycle.

ManelMoreno commented 5 years ago

Very good! Thanks for the fix and test.