search

eclipse / iottestware

Eclipse Public License 1.0
10 stars 12 forks source link

Test report in junit format #20

Open troian opened 4 years ago

troian commented 4 years ago

Hi, it may not be a right repository to create on the issue in so please direct if it should be moved somewhere.

We're currently automating builds on VolantMQ an open-source MQTT server with the goal to run iottesware on each commit.

That would be amazing if iottestware framework could generate reports in some widely used formats like junit. Unfortunatelly we couldn't find anything like that in the official documentation

If that's is something we could help to implement we would be happy to help.

alexanderkaiser commented 4 years ago

Hi @troian, I'm glad you are interested in using the iottestware.

That would be amazing if iottestware framework could generate reports in some widely used formats like junit. Unfortunatelly we couldn't find anything like that in the official documentation

The IoT-Testware is missing a proper solution for generating reports, which is indeed a very useful feature. However, the TTCN-3 runtime Eclipse Titan, which we are using has an integrated tool called repgen (chapter 5.4 in Eclipse Titan Userguide). Actually what repgen does, is generating the report in an additional step from generated log-files. Because of the fact, that the logs are structured and fairly easy to parse you can easily build your own report generator in a similar manner.

We have already done something similar in the Dashboard, where we use the stdout (which is very similar to the logs) to generate rudimentary PDF reports. For this reason, I have also translated the TDL Test Purposes into a separate JSON file for easier access, which might come handy while generating reports.

Please let me know if you need further input.

troian commented 4 years ago

Hi @alexanderkaiser Thank you for fast response and suggestions. Often logs are subject to change to so parser might get broken. Or logs structure is frozen?

rethy commented 4 years ago

Hi all,

actually Titan logger is built on a plug-in principle, with an internal plugger API. The text logger is "just" the default logger plug-in, but Titan also has a Junit logger plug-in (actually 2). They are described in clause 7.2.2. of Titan's Reference Guide https://www.eclipse.org/downloads/download.php?file=/titan/Titan_docs6_6_1.zip

They have not been touched for some time now, so maybe would need some updates to work with latest junit tools. Probably worth to try. @troian, would it solve your problem?

troian commented 4 years ago

@rethy Thank you a lot for reference. We'll definitely give it a try.