#717 Updated nunit to 3.x to make it work in VS2017 again

AdamsLair / duality

a 2D Game Development Framework

https://adamslair.github.io/duality

MIT License

1.41k stars 290 forks source link

#717 Updated nunit to 3.x to make it work in VS2017 again #717

Closed Barsonax closed 5 years ago

Barsonax commented 5 years ago

ToDo:

[x] Updated nunit 3.x
[x] Verify that tests can be executed again in visual studio 2017
[x] Fix remaining failing tests due to breaking changes between nunit 2.x and 3.x
[x] Verify AppVeyor test execution still works as expected
[x] Verify NightlyBuilder tool is still able to read test results
[x] Remove old "start test project using nunit UI" settings that have been superseded by VS integrated test discovery for a while now, once it works again
[x] Final clean up and review of code changes

Work in progress to fix to issue #716

ilexp commented 5 years ago

Do you want assistance in porting / fixing the remaining tests to the new nunit in this PR branch?

Barsonax commented 5 years ago

Would be appreciated as iam not familiar with the tests themselves. I believe the update broke something that is related to the setupfixture which in turn causes some tests to fail.

ilexp commented 5 years ago

I'll take a look as soon as I manage.

Edit: Updated the PRs summary to include some more ToDo items we need to check during the update.

ilexp commented 5 years ago

Checked out the PR branch, and VS discovers all tests as expected in the Test Explorer window, but fails to run any of them.

This is the output I'm getting:

[13/04/2019 14:18:42 Informational] ------ Discover test started ------
[13/04/2019 14:18:43 Informational] ========== Discover test finished: 0 found (0:00:01.2818452) ==========
[13/04/2019 14:18:43 Informational] ------ Run test started ------
[13/04/2019 14:18:43 Informational] ========== Run test finished: 0 run (0:00:00.4048189) ==========

Seems there is some sort of dissonance between test discovery in the UI and for actually running them. Any idea what's missing?

Edit: Managed to solve the issue by doing a clean + rebuild, then restarting VS. Not sure why it didn't work before, but that seems fixed now. Proceeding to look into tests now.

ilexp commented 5 years ago

Progress

Fixed the remaining test failures.
Verified AppVeyor can still run the tests without issues.

ToDo

Verify NightlyBuilder tool is still able to read test results
Remove old "start test project using nunit UI" settings that have been superseded by VS integrated test discovery for a while now, once it works again
Final clean up and review of code changes

ilexp commented 5 years ago

Progress

Fixed NightlyBuilder build script with msbuild 15.0 / VS 2017.
Found that the NightlyBuilder tool is currently not able to read back test results, likely because it uses the wrong test runner.

ToDo

Fix NightlyBuilder test result readback and verify it works again.
Remove old "start test project using nunit UI" settings that have been superseded by VS integrated test discovery for a while now, once it works again
Final clean up and review of code changes

Barsonax commented 5 years ago

Hmm what is this nightlybuilder tool? Why not use the dotnet tooling? Nowadays it can even make nuget packages for you.

ilexp commented 5 years ago

Progress

Fixed test runner version used by NightlyBuilder.
Found that the test result XML file format has been changed between v2 and v3, needs some adjustments in readback.

ToDo

Fix NightlyBuilder test result readback and verify it works again.
Remove old "start test project using nunit UI" settings that have been superseded by VS integrated test discovery for a while now, once it works again
Final clean up and review of code changes

ilexp commented 5 years ago

Hmm what is this nightlybuilder tool? Why not use the dotnet tooling? Nowadays it can even make nuget packages for you.

It's essentially a legacy tool that was used for CI-like workflow before AppVeyor took over most of its tasks. It's still used for packaging as part of the AppVeyor CI build right now and should probably be replaced by out of the box tooling at this point.

Until it's gone, it should be maintained though for good measure.

(Ping me back in a different issue in case you're interested in helping ripping it out!)

Barsonax commented 5 years ago

Ah then it makes sense. Made issue about it so we can start gathering idea's: https://github.com/AdamsLair/duality/issues/718