search

microsoft / dotnet-framework-docker

The repo for the official docker images for .NET Framework on Windows Server Core.
https://hub.docker.com/_/microsoft-dotnet-framework
MIT License
701 stars 336 forks source link

Code coverage tool doesn't seem to work #1133

Closed Szaamaan closed 2 months ago

Szaamaan commented 4 months ago

Describe the Bug

CodeCoverage.exe (version 17.0.24162.516205) which is part of the latest image (sdk:4.8.1) doesn't appear to work. For example, attempting to run a simple -? to get the help documentation results in a "blank" statement (see below for details).

This is a problem for our pipelines which later utilize SonarQube, which uses CodeCoverage.exe for test result parsing. It's also possible our tests (using DotNetCoreCLI@2 and using --collect "Code coverage" as an argument for testing) don't generate the required properties, but this is more difficult to check (since our docker instances are destroyed as soon as they're done).

Steps to Reproduce

CodeCoverage.exe is deprecated. Use dotnet-coverage global tool or Microsoft.CodeCoverage.Console instead. Some features of CodeCoverage.exe will be removed in next release of Visual Studio.

Usage: CodeCoverage 


- Attempt to use the image in a DevOps pipeline using `SonarQubeAnalyze@5` task, which results in the following:

Microsoft (R) Coverage Collection Tool Version 17.0.24162.516205 Copyright (c) Microsoft Corporation. All rights reserved.

CodeCoverage.exe is deprecated. Use dotnet-coverage global tool or Microsoft.CodeCoverage.Console instead. Some features of CodeCoverage.exe will be removed in next release of Visual Studio.

Usage: CodeCoverage

Process returned exit code -1


### Other Information

Running `.\CodeCoverage.exe -?` from `mcr.microsoft.com/dotnet/framework/sdk:4.8-20240312-windowsservercore-ltsc2019` produces the expected result:

PS C:\Program Files (x86)\Microsoft Visual Studio\2022\TestAgent\Team Tools\Dynamic Code Coverage Tools> .\CodeCoverage.exe -? Microsoft (R) Coverage Collection Tool Version 17.0.24072.507204 Copyright (c) Microsoft Corporation. All rights reserved.

CodeCoverage.exe is deprecated. Use dotnet-coverage global tool or Microsoft.CodeCoverage.Console instead. Some features of CodeCoverage.exe will be removed in next release of Visual Studio.

Usage: CodeCoverage 

Collect coverage data to the given coverage file.
Usage: collect [options] [command line]
Options:
    /config:<path>
    [optional] Override default configuration with configuration file
    specified.

    /IIS
    [optional] Collect code coverage for web applications running
    under Internet Information Service (IIS). IIS Worker process
    (w3wp.exe) will be registered for collection and all the application
    pools will be restarted.

    /output:<path>
    [required] The file to output coverage data to.

    /session:<name>
    [optional] The name for the session.  If a session name is not
    provided, a unique session name will be used

    /verbose
    [optional] Turn on printing of diagnostic information during runtime.

Example:
    CodeCoverage collect /IIS /session:WebSession /output:MyWebApp.coverage

Shut down a logger with a given session name.
Usage: shutdown [options]
Options:
    /session:<name>
    [required] The name of the session to shut down

Example:
    CodeCoverage shutdown /session:WebSession

Analyze data from one or more coverage files and output XML.
Usage: analyze [options] <file1> [file2 [...]]
Options:
    /include_skipped_functions
    [optional] Includes skipped functions in the results.

    /include_skipped_modules
    [optional] Includes skipped modules in the results

    /output:<path>
    [optional] The file to output the analyzed coverage data to
    The default is to use standard output

    /test:<test id>
    [optional] The test identifier string to limit the coverage
    data to. This option can be specified multiple times to merge
    coverage data for different tests together

Example:
    CodeCoverage analyze /include_skipped_functions
    /include_skipped_modules /output:results.xml MyWebApp.coverage

### Output of `docker version`

Client: Version: 26.1.3 API version: 1.45 Go version: go1.21.10 Git commit: b72abbb Built: Thu May 16 08:34:37 2024 OS/Arch: windows/amd64 Context: default

Server: Docker Engine - Community Engine: Version: 26.1.3 API version: 1.45 (minimum version 1.24) Go version: go1.21.10 Git commit: 8e96db1 Built: Thu May 16 08:33:14 2024 OS/Arch: windows/amd64 Experimental: false


### Output of `docker info`

Client: Version: 26.1.3 Context: default Debug Mode: false

Server: Containers: 7 Running: 7 Paused: 0 Stopped: 0 Images: 22 Server Version: 26.1.3 Storage Driver: windowsfilter Windows: Logging Driver: json-file Plugins: Volume: local Network: ics internal l2bridge l2tunnel nat null overlay private transparent Log: awslogs etwlogs fluentd gcplogs gelf json-file local splunk syslog Swarm: active NodeID: uqe4tp0cnljsxotkjoj3yse2p Is Manager: false Node Address: 10.150.1.54 Manager Addresses: 10.150.1.48:2377 Default Isolation: process Kernel Version: 10.0 20348 (20348.859.amd64fre.fe_release_svc_prod2.220707-1832) Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.2461) OSType: windows Architecture: x86_64 CPUs: 16 Total Memory: 26GiB Name: SRV-DCK-P02 ID: 55e34e14-2dc8-4191-9f60-9a23b349fd70 Docker Root Dir: C:\ProgramData\docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine

lbussell commented 2 months ago

Hey @Szaamaan, as the warning suggests, CodeCoverage.exe is deprecated. This may have came in with the update to VS 17.10.

I took a look around and it seems like there are newer versions of SonarQube/SonarScanner released that no longer use CodeCoverage.exe, can you try those out? https://community.sonarsource.com/t/codecoverage-no-longer-reported-in-sonarqube/116079/16

Szaamaan commented 2 months ago

@lbussell The warning suggests the tool will be removed at some point. As it is, it's "there" but completely doesn't work. At which point one has to wonder why it's there at all? Or at least that's my understanding of how the "deprecated" word is typically used.

The new @6 SonarQube tasks are relatively new, so demanding all pipelines change to these new tasks immediately is unfeasible (we're getting there, but even this small change will need time before it's deployed across all pipelines).

So it still feels like a bug. Or, at the very least, the warning shouldn't be a warning and make it clear that the tool isn't just "deprecated" - it's flat out obsolete and returns a -1 error code.

lbussell commented 2 months ago

[Triage] @Szaamaan, that seems like a question for the Visual Studio team. The .NET Framework images simply ship whatever is in Visual Studio. You can report a problem at https://developercommunity.visualstudio.com/VisualStudio.