dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.67k stars 1.06k forks source link

Error while running dotnet test on Ubuntu-latest in GitHub actions #29742

Open vincent1405 opened 1 year ago

vincent1405 commented 1 year ago

Describe the bug

In my CI workflow on github build.yaml, I use the following steps :

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0 #fetch-depth is needed for GitVersion

      - name: Set up .Net 7.0.x
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: 7.0.x

      - name: Restore dependencies
        run: dotnet restore
        shell: bash

      - name: Build 
        run: dotnet build --configuration Release --no-restore
        shell: bash

      - name: Test
        run: dotnet test --no-build
        shell: bash

The Test step fails with the following message:

|2023-01-04T16:43:05.0230657Z Test run for /home/runner/work/test-dotnet-test-in-ci/test-dotnet-test-in-ci/TestProject.Tests/bin/Debug/net6.0/TestProject.Tests.dll (.NETCoreApp,Version=v6.0)
 | 2023-01-04T16:43:05.1893117Z Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
 | 2023-01-04T16:43:05.1893862Z Copyright (c) Microsoft Corporation.  All rights reserved.
 | 2023-01-04T16:43:05.1969793Z
 | 2023-01-04T16:43:05.2761489Z The argument /home/runner/work/test-dotnet-test-in-ci/test-dotnet-test-in-ci/TestProject.Tests/bin/Debug/net6.0/TestProject.Tests.dll is invalid. Please use the /help option to check the list of valid arguments.
 | 2023-01-04T16:43:05.3767690Z Test run for /home/runner/work/test-dotnet-test-in-ci/test-dotnet-test-in-ci/TestProject.Tests/bin/Debug/net7.0/TestProject.Tests.dll (.NETCoreApp,Version=v7.0)
 | 2023-01-04T16:43:05.5347231Z Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
 | 2023-01-04T16:43:05.5351642Z Copyright (c) Microsoft Corporation.  All rights reserved.
 | 2023-01-04T16:43:05.5432911Z
 | 2023-01-04T16:43:05.6517255Z The argument /home/runner/work/test-dotnet-test-in-ci/test-dotnet-test-in-ci/TestProject.Tests/bin/Debug/net7.0/TestProject.Tests.dll is invalid. Please use the /help option to check the list of valid arguments.
 | 2023-01-04T16:43:05.7877738Z ##[error]Process completed with exit code 1.

To Reproduce

Please visit https://github.com/vincent1405/test-dotnet-test-in-ci

Exceptions (if any)

No Exception thrown, just an error message:

The argument /home/runner/work/test-dotnet-test-in-ci/test-dotnet-test-in-ci/TestProject.Tests/bin/Debug/net7.0/TestProject.Tests.dll is invalid. Please use the /help option to check the list of valid arguments.

Further technical details

Result of dotnet --info

 ##[group]Run dotnet --info
 dotnet --info
 shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
 env:
   DOTNET_ROOT: /usr/share/dotnet
 ##[endgroup]
 .NET SDK:
  Version:   7.0.101
  Commit:    bb24aafa11

 Runtime Environment:
  OS Name:     ubuntu
  OS Version:  20.04
  OS Platform: Linux
  RID:         ubuntu.20.04-x64
  Base Path:   /usr/share/dotnet/sdk/7.0.101/

 Host:
   Version:      7.0.1
   Architecture: x64
   Commit:       97203d38ba

 .NET SDKs installed:
   3.1.120 [/usr/share/dotnet/sdk]
   3.1.202 [/usr/share/dotnet/sdk]
   3.1.302 [/usr/share/dotnet/sdk]
   3.1.425 [/usr/share/dotnet/sdk]
   5.0.104 [/usr/share/dotnet/sdk]
   5.0.214 [/usr/share/dotnet/sdk]
   5.0.303 [/usr/share/dotnet/sdk]
   5.0.408 [/usr/share/dotnet/sdk]
   6.0.403 [/usr/share/dotnet/sdk]
   7.0.100 [/usr/share/dotnet/sdk]
   7.0.101 [/usr/share/dotnet/sdk]

 .NET runtimes installed:
   Microsoft.AspNetCore.App 3.1.4 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 3.1.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 3.1.20 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 3.1.31 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 5.0.4 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 5.0.9 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 5.0.17 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 6.0.11 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 7.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.AspNetCore.App 7.0.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
   Microsoft.NETCore.App 3.1.4 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 3.1.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 3.1.20 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 3.1.31 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 5.0.4 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 5.0.9 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 5.0.17 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 6.0.11 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 7.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
   Microsoft.NETCore.App 7.0.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

 Other architectures found:
   None

 Environment variables:
   DOTNET_ROOT       [/usr/share/dotnet]

 global.json file:
   Not found

 Learn more:
   https://aka.ms/dotnet/info

 Download .NET:
   https://aka.ms/dotnet/download
vincent1405 commented 1 year ago

I have just found a workaround: If I remove the --no-build argument, the tests run ie

dotnet test

instead of

dotnet test --no-build

NB: It even works with the --no-restore flag, the error seems to come from the dll being tested directly, without having been built by the test step.

vincent1405 commented 1 year ago

Not sure if this issue can be associated with: https://github.com/dotnet/sdk/issues/3684

ericsampson commented 1 year ago

I'm having the exact same issue now, whereas my GitHub Actions workflow used to work just fine.

Any idea what's going on?

fifi98 commented 12 months ago

Having the exact same issue. Once I add the --no-build argument it fails with the "The argument ...dll is invalid"

EDIT: found the mistake. are you still getting that error if you do dotnet test --configuration Release --no-build?

abdulwahid211 commented 10 months ago

@fifi98 is right, if you do dotnet test --configuration Release --no-build should work because in your build step you are setting the configuration for release dotnet build --configuration Release --no-restore

lucasoares commented 7 months ago

For me the error persists even building with dotnet build --configuration Release --no-restore and testing with dotnet test --configuration Release --no-build and I honestly don't want to remove the --no-build tag since I run different types of tests in different jobs on my CI and I don't want it building the project multiple times.

I'm trying to use ubuntu-latest instead of windows on my CI but this is proving to be very hard.

Edit:

I'm building using a solution filter (that also includes all the tests projects) but I'm running dotnet test in the test project csproj only, not the solution filter itself. Maybe this is related?

Edit2:

I tried to execute tests using the solution filter and added a --filter to filter which tests I want to execute, but the execution time for this is very high. The job that previously was running with 6s for unit tests now is taking 4~5 minutes even with --no-build, probably because it is analyzing all projects and searching for tests that only exists in two other projects (this solution filter have many and many projects).

kikutano commented 6 months ago

I have just found a workaround: If I remove the --no-build argument, the tests run ie

dotnet test

instead of

dotnet test --no-build

NB: It even works with the --no-restore flag, the error seems to come from the dll being tested directly, without having been built by the test step.

This worked for me!

lucasoares commented 3 months ago

I have just found a workaround: If I remove the --no-build argument, the tests run ie

dotnet test

instead of

dotnet test --no-build

NB: It even works with the --no-restore flag, the error seems to come from the dll being tested directly, without having been built by the test step.

This worked for me!

Yes, it works but it is not a viable workaround for 90% of the cases. If you have steps in your CI that first build the project and then runs the test, you will end up building everything twice, wasting resources and time.

baronfel commented 3 months ago

I want to see proof that this ever worked for folks. Setting a configuration of Release impacts the output paths that dotnet build and dotnet test use to locate the DLLs that should be tested. I can think of no actual scenario where doing a build in the Release Configuration and then testing without an explicit Configuration (which defaults to Debug) would work when --no-build is specified. The best way to prove exactly what's happening is with an MSBuild Binlog.

lucasoares commented 3 months ago

I want to see proof that this ever worked for folks. Setting a configuration of Release impacts the output paths that dotnet build and dotnet test use to locate the DLLs that should be tested. I can think of no actual scenario where doing a build in the Release Configuration and then testing without an explicit Configuration (which defaults to Debug) would work when --no-build is specified. The best way to prove exactly what's happening is with an MSBuild Binlog.

But what you're describing is not what is hapenning for me.

For me the error persists even building with dotnet build --configuration Release --no-restore and testing with dotnet test --configuration Release --no-build

Sadly I can't reproduce it anymore because I don't have access to the project I was testing.

nosalan commented 1 day ago

doing a build in the Release Configuration and then testing without an explicit Configuration..

@baronfel But we do specify the explicit configuration for both build and test yet the problem is still there. These are my commands: dotnet restore dotnet build --configuration Release --no-restore dotnet test --configuration Release --no-build

dotnet test still complains that the dll parameter is invalid.

Plevi1337 commented 16 hours ago

I've managed to get it working by specifying both the runtime and the configuration on both the build and test command

RUN dotnet build {TEST PROJ DIRECTOR}/{TEST PROJ CSPROJ} \
  --no-restore \
  --configuration Debug \
  --runtime linux-x64

CMD ["dotnet", "test","--runtime", "linux-x64", "--no-restore", "--no-build", "--logger:trx", "--results-directory", "/testresults", "--collect:\"XPlat Code Coverage\""]