search

pytest-dev / pytest-html

Plugin for generating HTML reports for pytest results
Other
704 stars 235 forks source link

pytest-html 4 performance regression #684

Open amezin opened 1 year ago

amezin commented 1 year ago

My test suite contains ~4k tests (heavy parametrization) with logs and sometimes screenshots.

The report generated by the latest stable version of pytest-html opens much faster than the report generated by 4.0.0rc4

Report generated with pytest 3.2.0 loads in 5-10 sec Report generated with 4.0.0rc4 takes about 40 sec

Firefox 114.0, Linux

Similar difference when you change sorting order, etc.

3.2.0 report: fast.zip 4.0.0rc4 report: slow.zip

BeyondEvil commented 1 year ago

Interesting, thank you for reporting!

I was expecting a slight performance decrease due to client-side rendering. But not that significant.

We have pagination on the roadmap, which should alleviate the loading. Not sure if it will help with sorting tho.

narmaku commented 1 year ago

That's great @BeyondEvil . I think pagination could possibly help as long as the rendering is scoped and done dynamically and not everything at the beginning of loading the HTML page/file.

In our case we have 10k test case results, and the rendering is very slow (10-12 secs) the first time you open the report. Even if you pass the "visible" param to hide e.g. passing tests, it seems that they will still render the first time you open the page (although remaining hidden).

Once they are hidden and page fully loads, the performance goes back to normal, as long as most of the tests are hidden. I will see if I can remove any confidential data from one of the reports and attach it here.

We are using pytest-html==3.2.0 though.