Azure DevOps pytest-cov option to inline CSS

[romanzenka]

Summary

Implement an option for pytest-cov that produces simpler, safer HTML with no external CSS that would be stripped by tools like Azure DevOps.

Azure DevOps by default strips "unsafe CSS", breaking the pytest-cov output to the point that one cannot tell which lines are covered or not. This makes the tool dramatically less useful when ran automatically in a CI server. A feature to make a self-contained, simplified .html file that can be safely embedded by 3rd party tools would alleviate this issue.

The only solution I am aware of is to install a node module that is able to re-write HTML and patch the CSS, which means one's python project would have to also use node.js and install npm modules. Since pytest-cov is in control of the report format, it would be easiest if it could output such "stripped" HTML directly.

[nedbat]

How does Azure determine that CSS is unsafe?

[romanzenka]

I do not know the exact functionality. It might be simply that all external CSS is deemed unsafe.

All I have found is this article:

https://developercommunity.visualstudio.com/t/code-coverage-report-on-build-should-not-strip-sty/365580

There is a person who inlines CSS into the HTML page using an npm module. I can experiment more and see if I can figure this out. I can try modifying the code to inline the most important styles that are getting lost - red/green for coverage - and see if it works.

Here is an example of what I am seeing. It is impossible to distinguish covered/not in this output:

[nedbat]

I wonder if something else is going on. The stylesheets are served from the same place as the HTML files. There's no reason to mistrust the stylesheet, but to allow the styles if they are inlined. Can you file a bug report with Azure DevOps to ask why this is happening?

[romanzenka]

@nedbat Microsoft is aware, this has been reported a few times, example here:

https://developercommunity2.visualstudio.com/t/Code-coverage-report-seems-brokenmiss/1149058?space=21&entry=problem

Microsoft takes the "this is a feature for your safety" stance here, so I am afraid chances of them resolving this are low.

[nedbat]

Have you confirmed that the styling is right if the CSS is inlined? It's not complicated to do, a small Python program could do it without involving npm.

[romanzenka]

I have not, I have relied on the npm tool using this trick, but I can do so before we proceed.

[ryanguo42]

From my test, AzureDevOps removed the following line: <main id="source"> Seems tag <main> is not allowed and get removed. A workaround is to change <main id="source"> to <div id="source"> by sed or similar tool. Then AzureDevOps test coverage will show expected result in its coverage result.

Is it possible to change the <main> tag to <div> tag for capability consideration? Thanks.

[mustafmst]

Is there any solution for bad formatting of the report on Azure DevOps?