teemtee / tmt

Test Management Tool
MIT License
85 stars 128 forks source link

`tmt tests export -h nitrate` should run checks before for all selected tests first #1984

Open comps opened 1 year ago

comps commented 1 year ago

When attempting to export 54 new tests to Nitrate, tmt fails with:

$ tmt tests ls | wc -l
54

$ git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

$ tmt tests export -h nitrate --create
Test case 'TC#0615155' created.
Append the nitrate test case id.
summary: /CoreOS/scap-security-guide/hardening/anaconda/C2S
components: scap-security-guide
tags: fmf-export
estimated time: 4h
automated: auto
status: CONFIRMED
arguments: PYTHONPATH=../../lib PROFILE=C2S
Structured Field: 
description: Remediates VM via anaconda %addon, scans via oscap
hardware: keyvalue = HVM=1
hostrequire = memory>=4000
fmf id:
url: https://github.com/RHSecurityCompliance/contest.git
name: /hardening/anaconda/C2S
Add migration warning to the test case notes.
Append the ID c7c35b94-ce61-43d5-a784-c60730e152a1.
2 Markdown files found in the current working directory.
Manual steps couldn't be exported
Test case 'TC#0615155' successfully exported to nitrate.
Can't export due 'Uncommitted changes in hardening/anaconda/main.fmf'.
Use --ignore-git-validation on your own risk to export regardless.

It seems that the uncommited-changes check is performed for each test export separately, instead of at the beginning of the entire export operation.

lukaszachy commented 1 year ago

Thanks for the feedback. You are right - it makes more sense to check first and only then do the export.

Good thing is that extra-nitrate is written to the disk as soon as case is created so you are not creating too many cases (as export fails with new issue each time you fix & rerun)

lukaszachy commented 1 year ago

Note to self: Check if polarion export runs the check - if it does it suffers from same behaviour