search

kbot7 / NgrokAspNetCore

Integration of Ngrok with the AspNetCore pipeline. Tools to automatically create ngrok tunnels on application startup
MIT License
25 stars 10 forks source link

Run unit tests on CI build and wire up to results #29

Open kbot7 opened 4 years ago

kbot7 commented 4 years ago

Setup unit test projects to run on the CI github action and publish the results in the artifact drop

kbot7 commented 4 years ago

I signed the repo up for coveralls.io.

I've added the coveralls token to GitHub secrets with the key name COVERALLS_TOKEN

ewisted commented 4 years ago

Minor progress made on this issue today. I added diagnostic logging to the dotnet test command withing the ci-build and found the core issue. There is a line that appears when running locally or on my CentOS box that appears in the log files-

[coverlet]Instrumented module: '/home/repos/NgrokAspNetCore/test/Ngrok.AspNetCore.UnitTests/bin/Debug/netcoreapp3.1/Ngrok.AspNetCore.dll'

This line is missing from the logs pulled from the build artifacts. I searched a few issues on the coverlet GitHub repo and found some other people were also getting this issue, specifically this issue. They are getting the same results as us, but when running a deterministic build. Upon checking the csproj files for Ngrok.AspNetCore and Ngrok.ApiClient, I found both had the following lines-

<Deterministic Condition="'$(GITHUB_ACTIONS)' == 'true'">True</Deterministic>
<ContinuousIntegrationBuild Condition="'$(GITHUB_ACTIONS)' == 'true'">True</ContinuousIntegrationBuild>
<DeterministicSourcePaths Condition="'$(GITHUB_ACTIONS)' == 'true'">True</DeterministicSourcePaths>

This points to the most likely cause of the test coverage results working locally (Windows) and on my CentOS box but not in CI is the conditional deterministic build condition on GitHub Actions. I attempted to follow the example deterministic build example, but after a few attempts to get it working, I got stuck at the following error-

/opt/hostedtoolcache/dncs/3.1.101/x64/sdk/3.1.101/Roslyn/Microsoft.Managed.Core.targets(104,5): error : SourceRoot items must include at least one top-level (not nested) item when DeterministicSourcePaths is true [/home/runner/work/NgrokAspNetCore/NgrokAspNetCore/test/Ngrok.AspNetCore.UnitTests/Ngrok.AspNetCore.UnitTests.csproj]

There's some light documentation on this error here, but it hasn't proved helpful in getting this working as of yet. I will pick this up again later. Feel free to pick at it in the meantime @kg73

kbot7 commented 4 years ago

@ewisted The test coverage errors were related to the dotnet sdk version the CI build was using. It used to be on 3.1.101. According to this coverlet issue, the issue was resolved in dotnet sdk 3.1.300+, so I upgraded our CI pipeline to use 3.1.401 and now it works correctly. I verified we are getting test coverage pushed to coveralls.io.

Feel free to drop your branch and cut a new one from the latest in develop. You'll still need to add the coveralls.io badge to the readme, and add test coverage to the (pre)release build.