JetBrains / TeamCity.VSTest.TestAdapter

Apache License 2.0
30 stars 14 forks source link

TeamCity.VSTest.TestLogger.dll not included after dotnet publish #21

Closed mexisd closed 6 years ago

mexisd commented 6 years ago

.NetCoreCli: v2.1.202

Given a .NetCore project that references

<PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.10" />

after dotnet publish {projectName}.csproj, the assembly TeamCity.VSTest.TestLogger.dll is not included in the published artifacts.

As a result if I use dotnet vstest {projectName}.dll to run my tests from TeamCity, no report is added. TeamCity will just report Success which is not very useful, instead of Tests passed: {numberOfTests}.

If I copy the file manually to the output before running my tests I get the correct results on TeamCity.

Am I doing something wrong? To me it seems that file should be included.

mhail commented 6 years ago

After digging through TeamCity.VSTest.TestAdapter.props the file is only copied when IsUnderTeamCity is set to true this can be faked from the dotnet command line using -p:IsUnderTeamCity=true or adding a property definition to the project file.

Ex: command line is dotnet test ... --test-adapter-path:. --logger:TeamCity -p:IsUnderTeamCity=true

NikolayPianikov commented 6 years ago

@mexisd To run tests using this adapter you should use the dotnet CLI command dotnet test to run tests for a project or a solution not for an assembly. In this case dotnet CLI runs restore, build and vstest targets.

To run tests for specific testing assembly you should use the 'dotnet vstest' command using dotnet CLI plugin or specifying the correct part to the adapter assembly by additional arguments /Logger:teamcity /TestAdapterPath:somePath.

NikolayPianikov commented 6 years ago

@mhail There are no any reasons to use IsUnderTeamCity under a TeamCity agent process because of this property is always true for this case.

<IsUnderTeamCity Condition=" '$(TEAMCITY_PROJECT_NAME)' != '' or '$(TEAMCITY_VERSION)' != '' ">true</IsUnderTeamCity>

TeamCity agent process always has environment variables TEAMCITY_PROJECT_NAME and TEAMCITY_VERSION.

mexisd commented 6 years ago

@mhail @NikolayPianikov Thanks for your responses. I will try the -p:IsUnderTeamCity=true and come back to you.

I am using the dotnet vstest but because the TeamCity.VSTest.TestLogger.dll is not in the artifacts I don't get results on TeamCity. For now I added the DLL manually and the reporting works.

I would expect when I do the dotnet publish to have the DLL in the artifacts. Is that assumption wrong?

Essentially, these are post deployment tests. I don't want to re-compile the whole project. So, in the build process of the application under test, a functional-tests.zip folder will be created with the outcome of the dotnet publish for the functional tests project. There the TeamCity.VSTest.TestLogger.dll is missing. Later another through another TC configuration, the artifacts will be downloaded through Artifact Dependency and the dotnet vstest command is run.

As I mentioned no results will be recorded. After adding the file in the artifacts I get results. If I use the dotnet test {projectName}.csproj I get correct results. But I don't want to re-build and want to use the existing artifacts.

NikolayPianikov commented 6 years ago

@mexisd Are you using the dotnet CLI plugin to run dotnet vstest command?

mexisd commented 6 years ago

@NikolayPianikov Good question. To not have a dependency on a specific agent the CLI, for the version we want, gets downloaded locally, during the build and that's the one being used.

NikolayPianikov commented 6 years ago

@mexisd From the TeamCity command line runner you could run tests like: dotnet vstest tests.dll /Logger:teamcity /TestAdapterPath:%teamcity.tool.TeamCity.Dotnet.Integration.DEFAULT%/vstest15

NikolayPianikov commented 6 years ago

@mexisd if you have any questions feel free to reopen it