danielpalme / ReportGenerator

ReportGenerator converts coverage reports generated by coverlet, OpenCover, dotCover, Visual Studio, NCover, Cobertura, JaCoCo, Clover, gcov or lcov into human readable reports in various formats.
https://reportgenerator.io
Apache License 2.0
2.58k stars 281 forks source link

Report Generation takes too many resources #608

Closed OrgesKreka closed 1 year ago

OrgesKreka commented 1 year ago

My coverage.xml file ( I can share if you need it ) is 126MB it contains 12k+ classes and takes more or less 7 minutes to be generated. But when I try to generate the report it takes more than 20 minutes ( I didn't let to finish ) and more than 10GB of memory. Is there any way to generate a simple MarkdownSummaryGithub because I use the GitHub action and I cannot use it this way.

Thanks

danielpalme commented 1 year ago

I would like to investigate this further next week. Can you share your coverage xml file?

Can upload it to GoogleDrive/OneDrive/Dropbox and send me the link? You can sent it confidentially by email: reportgenerator@palmmedia.de

danielpalme commented 1 year ago

Thanks for your email. I will have a look next week.

danielpalme commented 1 year ago

Could you please try again with the new release 5.1.22? I made some optimizations to significantly reduce the execution time. On my laptop memory usage never exceeded 900 MB. But let's see if this is still an issue with the new release.

OrgesKreka commented 1 year ago

@danielpalme Sorry for the late reply but I was on vacation. I tried today with the new version and as you can see from the screenshot now it consumes around 7GB and takes more than 10minutes to finish ( I killed it after 10 minutes ) image

danielpalme commented 1 year ago

Thanks for testing again.

I just tested myself with your coverage file and can confirm that the memory usage is pretty high: image

But after 60 seconds the coverage file is processed.

I will investigate this within the next days.

danielpalme commented 1 year ago

Your codebase contains ~3000 classes that are named *ArrayOf*.

The corresponding files seem to be very long (~150000 lines of code).

If I remove these classes with this filter argument -classfilters:-*ArrayOf* the memory usage drops to 2.4GB and execution time is about 30 seconds.

image

I guess that those large files are generated automatically by some tool. Maybe you can ignore them for your coverage report with the command line parameter: -classfilters:-*ArrayOf* `

OrgesKreka commented 1 year ago

Thank you very much for the debug, yes you are right. I have some old wsdls that have created their Reference.cs and one of those was very large. Since I could not exclude the Reference.cs from the coverlet I excluded them during the report generation, but again now for me it takes around 9 minutes to be generated. Below you can see the result: image

Is there anything to do to decrease the generation time?

danielpalme commented 1 year ago

Is there anything to do to decrease the generation time?

If you only want the MarkdownSummaryGithub report, you should not specify any other report types.

Your command line parameters should look like this: reportgenerator -reports:... -reporttypes:MarkdownSummaryGithub -classfilters:-*ArrayOf*

On my machine this command takes about 30 seconds. I don't know why it's that much slower on your machine.

Some potential issues:

OrgesKreka commented 1 year ago

@danielpalme thank you very much for your help! It looked like the issue was again related with the classes generated by the wsdl-s. I excluded the assembly that contains those classes and now it takes around 30s to generate the report. Wishing you all the best!