[report] option to do a backup report

[jencce]

When reporting to Polarion or ReportPortal, there could be an outage going on and other issues happending which leads the reporting step to fail. Then the whole plan fails, without any report data left.

If there is an option to do a backup reporting, for example a junit report, when the first option fails, we still have something useful to look at, which would be great.

After a job running long to the end, it's a pity not having any report.

[happz]

You may specify multiple report phases:

report:
  - how: junit
  - how: reportportal

Both should run, junit first, then reportportal, and even if reportportal fails, you should still have artifacts saved by junit.

[jencce]

That's great to know!

By the way, if I have the tmt run dir, aka /var/tmp/tmt/run-xxx, saved, can I run tmt to only get a report without executing all tests again?

[happz]

I believe so, tmt -vv run --id /var/tmp/tmt/run-xxx report

[jencce]

You may specify multiple report phases:

report:
  - how: junit
  - how: reportportal

Both should run, junit first, then reportportal, and even if reportportal fails, you should still have artifacts saved by junit.

Got error:

Invalid yaml syntax: Failed to parse '/root/tools/tmt-fs-errata/plans/run/linux.next-20240528.fmf'.
while constructing a mapping
  in "<unicode string>", line 41, column 9
found duplicate key "how" with value "reportportal" (original value: "junit")
  in "<unicode string>", line 43, column 9

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

[jencce]

I believe so, tmt -vv run --id /var/tmp/tmt/run-xxx report

Got an unexpected result, lots of plans in cwd got run and rsynced lots of stuff which ran out of disk space finally.

[jencce]

$ tmt --version tmt version: 1.34.0.dev8+g24d86c9c

[happz]

You may specify multiple report phases:

report:
  - how: junit
  - how: reportportal

Both should run, junit first, then reportportal, and even if reportportal fails, you should still have artifacts saved by junit.

Got error:

Invalid yaml syntax: Failed to parse '/root/tools/tmt-fs-errata/plans/run/linux.next-20240528.fmf'.
while constructing a mapping
  in "<unicode string>", line 41, column 9
found duplicate key "how" with value "reportportal" (original value: "junit")
  in "<unicode string>", line 43, column 9

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Can you share what exactly does /root/tools/tmt-fs-errata/plans/run/linux.next-20240528.fmf contain?

[jencce]

 40     report:                                                                     
 41         how: junit                                                              
 42         file: reports/local-junit.xml                                           
 43         how: reportportal                                                       
 44         project: "${TMT_PLUGIN_REPORT_REPORTPORTAL_PROJECT}"                    
 45         launch: "${TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH}.xfs"                  
 46 

[happz]

 40     report:                                                                     
 41         how: junit                                                              
 42         file: reports/local-junit.xml                                           
 43         how: reportportal                                                       
 44         project: "${TMT_PLUGIN_REPORT_REPORTPORTAL_PROJECT}"                    
 45         launch: "${TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH}.xfs"                  
 46 

Yep, I suspected that, it was reported by tmt, "found duplicate key "how" with value "reportportal" (original value: "junit")". Note that in my example, report is not a mapping with keys, but a list with entries that are mappings:

report:
  - how: junit
  - how: reportportal

# Not:
report:
  how: junit
  how: reportportal

Each entry describes one report phase.

[jencce]

 40     report:                                                                     
 41         how: junit                                                              
 42         file: reports/local-junit.xml                                           
 43         how: reportportal                                                       
 44         project: "${TMT_PLUGIN_REPORT_REPORTPORTAL_PROJECT}"                    
 45         launch: "${TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH}.xfs"                  
 46 

Yep, I suspected that, it was reported by tmt, "found duplicate key "how" with value "reportportal" (original value: "junit")". Note that in my example, report is not a mapping with keys, but a list with entries that are mappings:

report:
  - how: junit
  - how: reportportal

# Not:
report:
  how: junit
  how: reportportal

Each entry describes one report phase.

Oh, oops my fault! It works fine! Thanks and Closing!