Plain Text reporting functionality could be improved

[RobustoSystems]

First, thanks a lot for creating a fantastic tool! I started using LUnit a few weeks ago and like it very much.

The issue: Plain Text reporting does not seem to work properly for me. XML reporting works correctly.

I set up the Tools >> Reporting... settings as seen below:

Report path = C:\Users\petru.tarabuta\Work Folders\Desktop\LUnit Reports\LUnit Report - Hello World.txt

After I click on "Commit", I can see "Report generation disabled" in the bottom-left corner of the LUnit UI, as seen below.

Based on the contents of the first screenshot, shouldn't the UI display "Report generation enabled"?

The file at "C:\Users\petru.tarabuta\Work Folders\Desktop\LUnit Reports\LUnit Report - Hello World.txt" exists and is empty.

After running a group of three unit tests that are part of the same Unit Test class, on disk I can see that a new file named "LUnit Report - Hello World (1).txt" was created.

This file only contains the following line of text: "Fails: 0 Errors: 0 Skipped: 0".

It would be useful if:

• The file would contain more detailed information. For example, the file could contain a line for each Unit Test VI that executed. The text file contents could perhaps look similar to: "
  1. ABC.vi Passed 0.123 s
  2. DEF.vi Passed 0.051 s
  3. GHI.vi Failed 0.082 s
  4. JKL.vi Passed 0.56 s "
• LUnit could replace (overwrite) an existing file, rather than create a new file every time.
• The window where reporting options are configured would be much wider.

LUnit version: 1.6.2.17. LabVIEW version: 2024 Q3 64-bit. OS: Windows 11 64-bit

[RobustoSystems]

P.S. It may be better if the buttons of the reporting configuration window were named "Save" and "Cancel" instead of "Commit" and "Stop".

[antsundq]

Hey Petru,

Thanks for the feedback and happy to hear you like to tool.

When looking into this, I first thought the text reporting was broken. But then realized that the native text format only shows failed tests. My assumption being that passed tests are kind of un-interesting if you read a report file. Especially when you start having hundreds of tests.

Having said that, the reporting was designed to allow for extension through simple plugins. If you want to have a specific report format you'd want to implement, you just need to implement the Report Interface.lvclass: To further integrate it into LUnit, I'd need to make some small changes, but the support for plugins is there. Let me know if you are actually interested in this and I will write the required documentation for how to do it.

Honestly, I have not seen anyone use the reporting features through the UI before. I only use it for CI, and then with the xml-format. I'd be interested in understanding your workflow and what you use the text reports for? This is more out of curiosity and maybe we could figure out how to make this feature useful.

As you say, the configuration utility is not a great UI. I made a quick stab at improving these. If you want to make it even nicer, I'm happy to accept a pull request. It would also need to be updated to handle plugins, if someone feels like creating such. But until that day, I think it works well enough as it is.

Thanks again for the feedback!

/Anton

[RobustoSystems]

Hi Anton,

Thanks for the very quick and detailed reply.

It's great to hear that LUnit supports plugins - that's a very good design. Yes, I would be interested in creating a class that implements the Report Interface.lvclass.

I have not used LUnit through the CLI yet. I would like to try it in the near future.

My workflow consisted of using the LUnit UI to run a LUnit Test Class that contained three test VIs. Together with a colleague I started playing around with the reporting options, as we were considering making running unit tests a pre-requisite before code reviews. But we would be happy to do this through the CLI rather than the UI if it offers more comprehensive reporting.

I would be happy to make a few changes to the configuration utility UI. I am quite busy with work and life at the moment, but I aim to look into it.

Kind regards, Petru

[RobustoSystems]

I have created a fork of the repository and have had a quick look at the source code. It's impressive.

Can I ask a silly question: What is the name of the VI that represents the reporting configuration UI?

[antsundq]

Alright, I will have a look at it when I find some time.

The dialog is located here: astemes-lunit\source\User Interface\Execution UI\Private\Configure Reporting Dialog.vi

Please note that I will likely need to change this VI to enable loading of 3rd party plugins. Currently the xml and txt plugins are hardcoded. So if you want to work on the UI, don't let your fork live long before making a pull request or we will have conflicts. Alternatively, wait for me to get around implementing loading of reporting plugins.

Regarding the workflow, I think that using the UI gives the most detailed and easiest way to navigate the reports. You could run it through the cli (Sam has a GCli implementation on VIPM also if you prefer this), but I do not think it is better than the UI in any way (for humans).

/Anton

[antsundq]

Turns out it was very little needed to enable loading of custom reporting plugins. Just had to replace the hardcoded report names with dynamic enumeration.

When you want to start writing your own plugins, just get the latest version of LUnit from here and follow these instructions.

Let me know if you have any issues.

[antsundq]

Closing this issue for now, let me know if you run into any issues creating your custom reports.

[RobustoSystems]

Hi Anton,

Thanks for the update, and sorry for the delayed reply. Also sorry that I haven't made any changes to my LUnit fork yet - I have been busy at work and at home.

I am looking forward to meeting you at GDevCon tomorrow!

Kind regards, Petru

[antsundq]

No worries. Looking forward to catching up with you here in Stuttgart!