search

SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.24k stars 754 forks source link

Specflow Parallel Execution using MsTest not working #1439

Closed anandprasathm closed 3 years ago

anandprasathm commented 5 years ago

Specflow Parallel Execution using MsTest

SpecFlow Version:

Used Test Runner

Version number:

Visual Studio Version

Are the latest Visual Studio updates installed?

.NET Framework:

Test Execution Method:

<SpecFlow> Section in app.config

<specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="mstest.v2" /></specFlow>

Issue Description

Hi Friends, I am trying to run my Specflow Tests in Parallel using MsTest V2. Below is the Test Runner file that i am using to run my Specflow Tests in Parallel. I have this configured on Test Settings as well but still my tests are running one by one. Not sure if i am missing some other settings. Could someone please assist me on this?

Test Runner File:

<RunSettings>
<!-- MSTest adapter -->
  <TestAdaptersPaths>//MS Test Adapter Path</TestAdaptersPaths>
  <MSTest>
    <Parallelize>
      <Workers>0</Workers>
      <Scope>ClassLevel</Scope>
    </Parallelize>
  </MSTest>
</RunSettings>

Thanks in Advance.

SamJongenelen commented 5 years ago

Are you running in VS itself or via CLI? Did you make sure the .runsettings are used ?

(in VS, menu> test > test settings > select test settings file)

Are your tests in multiple classes or just one single? Did you enable the ' run tests in parallel' button in the VS test explorer?

anandprasathm commented 5 years ago

@SamJongenelen Thanks for getting back to me on this. I am running the tests on Visual Studio and i have .runsettings file configured as well. I have also enabled 'Run Tests in parallel' button as well. Since, i am using Specflow, all my tests start and ends in Hook class. Is there anything else that i am missing?

Thanks in Advance.

SabotageAndi commented 5 years ago

@anandprasathm You have worker set to 0. Is this limiting it? We execute in the SpecFlow 3 Test Suite the tests on 4 threads and they are working.

anandprasathm commented 5 years ago

@SabotageAndi I have set the workers to 4 but still my tests are running one by one. I have added the project files in the below clone URL. Could you please have a look at this and help me with my parallel execution? https://github.com/anandprasathm/MsTestV2.git

Thanks.

SabotageAndi commented 5 years ago

@anandprasathm If you have the problem with SpecFlow 3, I will have a look at it. We are about to release it in the next days. No time to analyse this in an old version.

anandprasathm commented 5 years ago

@SabotageAndi Thanks. I will check the same on Specflow version 3 and will come back if the issue still persists.

vaibhavvjy commented 5 years ago

@SabotageAndi I am also working in specflow facing issue in paralel execution not aware how to set runsetting and how to setup parallelize attribute. I am executing tests using command line .

Please help me with proper steps if any.

Parallel Execution

SamJongenelen commented 5 years ago

@vaibhavvjy I would recommend for you to get to know the mstest test engine/runners/settings/parallelism using plain unittests before moving to Specflow unittests.

KTJ-Demo commented 4 years ago

Any progress on this issue? I'm not able to get it working either..

anandprasathm commented 3 years ago

@SabotageAndi , With the current Specflow Version (3 and above) and MSTest V2, now i can able to run my Specflow Tests on Parallel.

But intermittently, i am getting the below error on the AfterStep of the last step of the testcase

System.ArgumentException: 'An item with the same key has already been added.'

Could you please help to resolve this?

SabotageAndi commented 3 years ago

@anandprasathm please open a new issue for this.

SabotageAndi commented 3 years ago

Original issue is solved => closing this one.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.