search

hibri / HttpMock

A library for creating Http servers on the fly in tests and stubbing responses
MIT License
128 stars 44 forks source link

Parallel execution of scenarios with HttpMocks #86

Closed pratishek closed 7 years ago

pratishek commented 7 years ago

I was assign one work to optimize the build process which was taking around 2 hours to complete. So i made changes to the code base to run the scenarios in parallel. In this due process i made changes to HttpMock repo to support session and concurrency. Now with these changes it brings down the test running time to 30 minutes.

With the current changes now httpmock will support:

  1. Concurrency.
  2. Session.
  3. Private/Shared Mocks (in the context of initializing a single HttpMock server with parallel running scenarios )
  4. WithNewContext support to individual scenarios. (Which will not break already running scenarios in parallel)

Thanks!

hibri commented 7 years ago

Hi @pratishek Thanks very much for this. Will look at this in detail tomorrow.

hibri commented 7 years ago

Hi @pratishek , Could you explain more about what you mean by Session? Could you please send separate PRs for the different features you've added please. It makes it easier to read the code. I'd like a separate PR for the concurrency work especially.

HTH

pratishek commented 7 years ago

@hibri ,

By session i mean to add capability to httpmock running multiple scenarios in parallel. So if we are trying to imitate a single API server with multiple mocks (it can be a case that the multiple scenarios are running for the same endpoint) . So to satisfy the use case of all these parallel running scenarios with their own mocks for the same endpoint and with the requirement to use WithNewContext multiple times during the execution as per their use case we have this session.

So for matching a scenario having it own mock with the requirement to call withnewcontext during the test execution, i put _handlers in a concurrent dictionary which will get identified during the respective OnResponse call by checking the session id in the request headers.

By default We have an empty guide session id for supporting existing httpmock behavior and the same can be used as the concept for Shared Mocks in parallel execution ( Setting it up in the BeforeTestRun, So that it can be used by all the scenarios where endpoint deliver same static/unchanged data during the course of test suite run). Private mocks/handlers will be residing in their own session id.

Concurrency is added in respect of maintaining the session itself in parallel execution context. In single test run case it wont be have any effect than the previous httpmock.

So a separate PR for concurrency and session wont be serve the purpose.

I was thinking if we can get it as separate nuget package itself then it wont be a concern for you as running the test in parallel execution unearthed more flaky test which wont go well with other users( if they don't have any intention for running it in parallel) . If you are okay with this then please suggest naming convention for that :grinning: .

hibri commented 7 years ago

Hi @pratishek You can always build it as a private Nuget package to suit your own needs. I think this will break for many users, and I'd like to keep the API interface a bit more friendly. I'm closing this PR for now. Let me know if you do get your flaky tests working. :)