dotnet / docs

This repository contains .NET Documentation.
https://learn.microsoft.com/dotnet
Creative Commons Attribution 4.0 International
4.22k stars 5.87k forks source link

--parallel flag, migrated from dotnet vstest, is missing in dotnet test documentation #35931

Open uladz-zubrycki opened 1 year ago

uladz-zubrycki commented 1 year ago

Issue description

According to the dotnet vstest documentation page, the command is superseded by the dotnet test command, which can now be used to run assemblies.

Which, I believe, was done as a result of https://github.com/microsoft/vstest/issues/1453 request in a scope of https://github.com/microsoft/vstest-docs/pull/196.

dotnet vstest is able to run tests from the assemblies in parallel if --parallel flag is provided. Per the referenced RFC, there was an intention to support the same behavior for the dotnet test command as well.

Success criteria (as per user ask)

  1. The "dotnet test" command can be used in place of the "dotnet vstest" command.
  2. The "dotnet test" command can accept a list of dlls
  3. Using "dotnet test" and targeting a list of dlls, there is an argument for "--parallel" which will run the tests from the dlls in parallel.
  4. Using "dotnet test" and targeting a dll, the inclusion or exclusion of the "--no-build" argument will have no impact on the behavior of execution.

My local testing confirms that --parallel flag is respected and tests are run in parallel; test from the assemblies are run sequentially otherwise. Note, that I'm talking parallel execution of the assemblies themselves and not the internal assembly-level of tests parallelization, which is handled by NUnit and its adapter for me.

However, there is no documentation for that feature at the dotnet test page and in the output of the dotnet test --help command, which is confusing.

  1. Is parallel execution of tests from assemblies supported by the dotnet test command officially, and it's safe to rely on that, or it's an implementation detail exposed accidentally?
  2. Should the documentation for dotnet test be updated to reflect that?

Target framework

dotnet --info output ```console .NET SDK: Version: 7.0.304 Commit: 7e794e2806 Runtime Environment: OS Name: Windows OS Version: 10.0.22621 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\7.0.304\ Host: Version: 7.0.7 Architecture: x64 Commit: 5b20af47d9 .NET SDKs installed: 3.1.100 [C:\Program Files\dotnet\sdk] 3.1.426 [C:\Program Files\dotnet\sdk] 6.0.100 [C:\Program Files\dotnet\sdk] 6.0.118 [C:\Program Files\dotnet\sdk] 7.0.302 [C:\Program Files\dotnet\sdk] 7.0.304 [C:\Program Files\dotnet\sdk] .NET runtimes installed: Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Other architectures found: x86 [C:\Program Files (x86)\dotnet] registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation] Environment variables: Not set global.json file: Not found ```
dotnet test --version output 17.6.3.22601
dhrumil0902 commented 2 months ago

@uladz-zubrycki I am at a similar cross-road, have you made a descision or got more information regarding this?

uladz-zubrycki commented 2 months ago

@dhrumil0902 as I remember, I've ended up using the flag and haven't ever rechecked it since then.