search

Azure / azure-sdk-for-cpp

This repository is for active development of the Azure SDK for C++. For consumers of the SDK we recommend visiting our versioned developer docs at https://azure.github.io/azure-sdk-for-cpp.
MIT License
178 stars 127 forks source link

Measure test coverage on Windows and consider adding to the CI #4050

Open ahsonkhan opened 1 year ago

ahsonkhan commented 1 year ago

We currently measure on Linux only, so only the curl transport paths are showing up. The WinHttp code coverage is missing.

Extracted from https://github.com/Azure/azure-sdk-for-cpp/pull/4015#discussion_r1002065122

Our code coverage CI leg only runs on linux/curl, so we don't see the coverage number for Windows-specific transport adapter files:

https://dev.azure.com/azure-sdk/public/_build/results?buildId=1934177&view=codecoverage-tab

image

https://github.com/Azure/azure-sdk-for-cpp/blob/53c9da155244b72ac90ad27f515d4ea1e7505a01/eng/pipelines/templates/stages/platform-matrix.json#L143-L149

https://github.com/Azure/azure-sdk-for-cpp/blob/53c9da155244b72ac90ad27f515d4ea1e7505a01/eng/pipelines/templates/stages/platform-matrix-live.json#L37-L43

cc @danieljurek, @vhvb1989, @LarryOsterman

danieljurek commented 1 year ago

Azure DevOps supports one set of code coverage statistics for an entire pipeline run. This forced us into the current configuration we have today.

https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-code-coverage-results?view=azure-devops#is-code-coverage-data-merged-when-multiple-files-are-provided-as-input-to-the-task-or-multiple-tasks-are-used-in-the-pipeline

To that end, we should probably focus on outputting code coverage in the logs but there wouldn't be an obvious way to bubble that information up and easily enforce it the way that we do with our linux/curl run today.