search

Qytera-Gmbh / QTAF

QTAF is a Java test framework based on TestNG and offers easy setup of Selenium and fast extensibility.
https://qytera-gmbh.github.io
MIT License
10 stars 0 forks source link

QTaf Logs #272

Open mzdih opened 6 months ago

mzdih commented 6 months ago

It is challenging for test analysts using QTaf to identify and analyze problems. This is partly because numerous warnings are displayed, which can be overwhelming for individuals without a technical background. Could you please eliminate such warnings? I have attached a screenshot displaying some of these warnings.

The client desires a feature that allows them to directly locate in the Logs the specific step where the test failed, as well as the test case number, which is currently not provided.

QtafLogsWarnings QtafLogsWarnings2

OliverHoenig commented 5 months ago

Thanks mzdih, for reporting. 

If I get you correctly, this issue groups two independent issues: 1) Warnings in logs 2) Log Steps and case number

I need more information on both points to avoid misunderstandings: 1) These warnings seemed to me to be justified. Have you already tried to follow the instructions of the warnings? Or are these warnings unjustified? What behaviour would you like to see from QTAF? (e.g. an option not to log warnings in general?)

2) "The client desires a feature that allows them to directly locate in the Logs the specific step where the test failed, as well as the test case number, which is currently not provided." --> "directly locate in the logs the specific step where the test faild": I'm not sure what is meant by this. If the qtaf.json logging.logSteps = ture was set, the steps should also be logged. The status of the steps is also displayed. --> In addition, the name of the step that was passed in the step annotation is logged. Doesn't this already solve the challenges described? If not: What behaviour would you like to see and why?

Thanks for your help

mzdih commented 5 months ago

  1. You're right about the warnings; they could be helpful for me as a developer to solve them. The issue is that the same logs will be used by the test analysts, who do not have a technical background, and those warnings could be distracting for them.

  2. Directly locating the problem in the logs means that if the client examines the logs, they could directly read something like: "Failed: Step 10: click save button: SQA-10" where they have a clear overview of the problem without needing to sift through all the warnings and the entire exception stack trace.

These are the client's wishes, and we should also discuss internally whether this approach makes sense for us and for QTAF.

OliverHoenig commented 5 months ago

Can this issue be closed or would you like to be able to switch off the warnings in QTAF? Personally, I would refrain from making this adjustable, as some warnings are quite justified and should be displayed in any case.

csvtuda commented 5 months ago

I don't think the warnings are related to QTAF. They are compile time warnings in @mzdih's test code, which they should probably fix themselves.

The NullPointerException however is a QTAF problem.