Uploading Test results to JAMA directly, not using Allure-Reporter manual features

[ManelMoreno]

In behalf of @joaquimvila , it has been proposed to evaluate the possibility of having the test results uploaded to JAMA, without manual actions using Allure-Reporter UI. That is, using directly the JAMA REST-API.

I (Manel) am ok with the proposal (in fact the original update used in QL4 was in this way), but my conditions here would be:

1. The username registering the results can not be an impersonal user. It shall be a team member.
2. The CI tool shall upload results just labelled/formal versions, to avoid having in JAMA hundreds of useless results.
3. Current UI and features can always be used, being NOT replaced by the proposal.
4. The direct upload satisfies the required information to be filled in a formal Test Run

[olgapuig]

The proposal is technically possible, but it requires a development in each project, and different for each technology. This means that the projects need to assign resources to do this direct connection with JAMA through the Rest-API. I think the LOE is not low.

From a user need and a project with 3 deadlines (Alpha, Beta and Final release), the manual upload of results into JAMA must be done just 3 to be compliant with the procedures, one for each delivery.

Allure-reporter was created to save time in the projects and be compliant at the same time. We have a unique tool for all the projects.

In any case, we can evaluate it to better understand it, and then make a decision together.

[joaquimvila]

The suggested improvement shouldn't imply doing things in a different way for each one of the technologies used at CSW. Moreover, considering that all projects converge to 2 shared report formats (JSON and XML), the resolution of this issue should start from these reports as input. This way, all projects could take advantage of this new feature easily.

One idea to tackle this without big effort could be the creation of an automated UI script. Triggered from the CI, the script could navigate to the current web page of the tool, fill in the required forms and perform the upload. Something similar to what we do with e2e tests with protractor, but with the difference that this wouldn't be a test, but an automated procedure.

This script could be guided by a configuration file containing all the information required to perform the upload in the desired manner. Thus, each project should only need to define the configuration file and trigger the script.

Some time ago, I implemented a proof of concept of this idea using NodeJS and Puppeteer. You can find it on the following link: https://github.com/systelab/allure-reporter-ci

Despite not being an everyday activity, this task is repetitive and error prone when performing it manually. Thus, integrating it as part of the continuous integration pipeline would avoid this kind of errors and speed up the generation of releases. Additionally, it would open the door to uploading results more frequently (i.e for each test package produced at the end of a sprint).

[olgapuig]

Good idea @joaquimvila, let's assess it!

[ManelMoreno]

Ok to implement it but again:

• Take into account the number of times that this is done in one year, the cost of the manual action, and the control/customization opportunity done by the person who uploads
• All the information must be passed programmatically before uploading the results (sometimes the field Actual Results is different depending on the execution or result (see TC Design Guidelines template).

This is not mandatory, just an improvement. TBD the implementation cost