urasandesu / Prig

Prig is a lightweight framework for test indirections in .NET Framework.
Other
117 stars 21 forks source link

TeamCity Support #88

Open ngjermundshaug opened 7 years ago

ngjermundshaug commented 7 years ago

Having difficulty making PRIG work under TeamCity (msbuild 2015)

[MSBuild output] ConnectorTests\ConnectorPrivateTest.cs(4,40): error CS0234: The type or namespace name 'Prig' does not exist in the namespace 'XXXXXXX.YYYY' (are you missing an assembly reference?) [C:\BuildAgent\work\e7ff6effff03f\xxxxx.yyyy.com\Tests\PrigTests\PrigTests.csproj]**

Can PRIG only be run in VS ?

urasandesu commented 7 years ago

I have never tested Prig in TeamCity, sorry. It seems too expensive for us who are open source developer.

However, Prig supports free CI environment like AppVeyor. The document possibly help you.

ngjermundshaug commented 7 years ago

TeamCity is totally free as long as you have less than 20 build configurations per server! We use the free version in many projects to take care of building, testing and deploying our code (along with Octopus).

You should really check it out - https://www.jetbrains.com/teamcity/download/

urasandesu commented 7 years ago

OMG :scream: I have never known that! I will try it and investigate your reporting problem. Thank you for the information!

andy250 commented 7 years ago

There are 2 issues actually:

  1. The prig test project won't compile on TC as @ngjermundshaug mentioned
  2. When we generate those assemblies locally (using VS), then commit those assemblies to code repository and reference them via "Add reference" in the prig test project then the whole thing compiles on TC. But another problem is that the test runner in TC does not enable the prig test adapter (or there is no test adapter at all). For this reason the tests are using original implementations instead of stubs. By enabling test adapter I mean clicking this: http://bit.ly/2jon8cz.

So we need a way to let TeamCity replace the implementations with stubs before running the tests. What actually this option in VS ("Enable Test Adapter for ...") does behind the scenes?

EDIT: TC supports installing custom plugins so I believe there should be a way to create such a test adapter (as a plugin).

andy250 commented 7 years ago

We have made it working with following steps:

  1. Prig generated assemblies and Prig native dll (that get installed as nuget packages but are not inthe nuget feed) are committed to code repository and then referenced manually with "Add reference" to proper projects
  2. Installed and registered Prig in VisualStudio which is running on the build agent
  3. Modify UnitTest build step not to use built-in test runner (e.g. VSTest), but instead use a CommandLine step. Reason is that prig needs to do some work before the tests are run. The command:

cd Tests\PrigTests\bin\Release prig run -process "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" -arguments PrigTests.dll

  1. Run the build and enjoy Prig on TeamCity

Thanks again for making Prig.

urasandesu commented 7 years ago

I'm temporarily relieved to hear that you made it working :relieved:

One of the things you might notice is that Prig generated assemblies -- we call them Prig Assembly -- are sensitive than you expected. It will break by a trifling matter. For example, applying Windows Update, installing .NET service pack and so on(This is the reason that we decide to generate Prig Assembly on build time). So, when the committed Prig Assembly is not worked, you have to rebuild and commit them again.

Please contact me if there is anything.

tjad commented 6 years ago

Just an added note, I've got prig running on our Jenkins build server. My basic steps are to get tests compiling/running on your buildserver from the commandline first. Once they are running, you should easily be able to adapt to your CI system - especially if it supports the CLI (Powershell/Cmd).

Be sure to reference the wiki for pointers! https://github.com/urasandesu/Prig/wiki/AppVeyor-Support