machine / machine.specifications

Machine.Specifications is a Context/Specification framework for .NET that removes language noise and simplifies tests.
MIT License
885 stars 178 forks source link

Support for CI integration and reports (Bamboo etc) #366

Open sonalsatpute opened 9 years ago

sonalsatpute commented 9 years ago

Team:

I am not sure if this is the right place to ask this question. We have set up a Bomboo Server and I am planing to use MSpec for writing unit-tests. What I want to do is to configure Bomboo to execute all the unit-tests/specifications. In case if the unit tests fail, server should send email with error details to all the users.

I have searched over internet but haven't found information on how to integrate MSpec with Bomboo to achieve above mentioned functionality.

Any help will be greatly appreciated,

Thanks in advance

agross commented 9 years ago

Do you know how to run programs on Bamboo CI?

sonalsatpute commented 9 years ago

Yes Alex, we can run the programs on Bamboo CI.

agross commented 9 years ago

Then run mspec.exe with your compiled spec assemblies and the --html <file> option. Define the HTML file as a build artifact (in TeamCity they're called artifacts, no idea about Bamboo).

What the Bamboo server uses to notify you about failed specs is beyond our control. Perhaps it's possible to attach certain build artifacts. Perhaps not. CI servers typically check the exit code of the programs run and error out /notify users when something returns exit code != 0.

In TeamCity you could have a second build config that uses the artifacts of the "test run" build and send e-mails with attachments yourself. Then again, TeamCity will also report test errors directly in its failure notification. Again, I have no idea about Bamboo.

sonalsatpute commented 9 years ago

Let me play a bit with Bamboo CI and will update you. Thanks for quick response.

ot-matt-sell commented 9 years ago

@sonalsatpute Any update on this? I'm also trying to configure bamboo CI to run mspec tests.

sonalsatpute commented 9 years ago

@ot-matt-sell :

I am not sure if this is the right way to configure it but below is the step how I have configured MSpec.

  1. Create the Stage (MSpec)
  2. Add the Job
  3. Add Task -> Select Command from list
  4. Click on Add New executable and give the path for you MSpec Console Runner
  5. In Argument test box C:\Users\sonal\bamboo-home\xml-data\build-dir\131073\SPRIN-TEST-JOB1\Build\MSpec\sprinkler.specifications.dll --html C:\Users\sonal\bamboo-home\xml-data\build-dir\131073\SPRIN-TEST-MSPEC\sprinkler.specifications.html (please change the path as per your folder structure)
  6. Save

Bamboo should be able to trigger the MSpec runner here is the some screen shot of my MSpec runner result. In case your tests fails Bamboo will send you email for same.

mspecx-log

Even though I have 2 test it says 0 test in total

mspecx-test-result

Where as Bamboo has very good in-built support for NUnit below is the screen-shot for reference I would like to to have something like this.

nunit_job_result_summery

nunit_test

When you click on any test you get the full details

nunit_test_details

danielmarbach commented 9 years ago

have no idea how bamboo works. But you need to look into the doco like https://developer.atlassian.com/display/BAMBOODEV/Test+Collection+and+Reporting and then write a reporter. Samples can be found here:

https://github.com/machine/machine.specifications.reporting/tree/master/Machine.Specifications.Reporting/Integration

Send a PR, I'll merge, release and there is your bamboo support :dancers:

dennywu commented 9 years ago

@sonalsatpute hi sonalsatpute, any update for this issue ? I'm also try configure run mspec test on bamboo. Now still confused. Can you help me ?

sonalsatpute commented 9 years ago

@dennywu : have you try the steps I have mentioned above on my comment on Apr 23 ?

dennywu commented 9 years ago

Tried it. But not working. I used window AMI, and change to another ebs. When i try checkout source again, i have got problem with user permission.

ccezarr commented 7 years ago

Do we have a nice solutiong to this?