Automated notification test failure template

[Werkov]

When there is a soft failure, the message posted into BZ neither openQA console dump may not contain information to uncover the failure cause. (Or it is well hidden for a person not so familiar with those reports.)

I suggest to modify record_soft_failure in such a way that the message would contain location of the actual code (similar to C assert). E.g link to the code

Soft fail at lib/sles4sap.pm:85, see [1]

[1] https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/7cb2bc0e2831c19b01777a0df019c5f50641125c/lib/sles4sap.pm#L85

Cc: @ricardobranco777 @okurz

[okurz]

while theoretically possible I see it challenging as it involves parsing of a deeper level than previously conducted, e.g. we would need to parse autoinst-log.txt

[Werkov]

I'm no Perl hacker, however, couldn't something like stack parsing be used to pinpoint the callsite?

[okurz]

That's not the problem. openqa-review so far only access higher levels of openQA. To get the details of a test failure openqa-review would need to read out files from failed jobs. Then showing the details within is not such a big problem. What you mentioned in before as well as "stack parsing" sounds like it could be easier done in the test backend, e.g. os-autoinst. This part here for openqa-review needs to be handled individually anyway.

[Werkov]

Ad original report, yes, having the report in BZ might be overkill inappropriate on second though. However, the report displayed after following the BZ link could give more information about the particular soft failure, is that the easier case?

[okurz]

Not sure what you mean with "BZ link". Do you mean the direct link to an openQA test run? This one should already show explicitly enough the soft failure reason. If you have recommendations to improve this please create a ticket for openQA itself, on https://progress.opensuse.org/projects/openqav3/issues/new

[Werkov]

Let's forget about BZ link.

I mean this message

It should show information about the file/line of the particular check, so that one can see what actually failed. Is that the openQA proper?