tvarchive / optimusTemplate

This is a cucumber jvm implementation template using the optimus mobile automation framework. If you like it, star it, help spread the word.
http://docs.testvagrant.com
66 stars 23 forks source link

Ability to execute tests at Feature level instead of scenario #33

Open sarathsp23 opened 5 years ago

sarathsp23 commented 5 years ago

Currently the setup and teardown is done on scenario level , which approximately takes around 1-2 mins each per scenario. In my case , say I run a test with around 20 scenarios, the actual test scenario execution takes around 5-7 mins , but another 8-10 mins goes into setup and teardown per scenario. Thats around double the actual test execution.

I have done a temporary workaround at my end by registering or de-registering the bot on feature level by adding few conditions in the starting steps.

But I was wondering if you could give this as an option, so we can reduce the execution time?

KrishnB commented 5 years ago

Hi Sarath,

Optimus runs each scenario in its own container thus reducing the flakiness but trade-off is extra time of setup and teardown. Your suggestion sounds really helpful, we will definitely take it as part of our backlog for future releases.

Jaspreet23 commented 5 years ago

Hi @sarathsp23 Can you share how you achieved doing the setup and tear down at the feature level. I am also facing the same issue and moreover in my case the IOS scripts doesn't execute in a suite as the webdriver agent gets installed and reinstalled after every scenario which causes the script to fail due "Xcode -Cannot create a session issue"

sarathsp23 commented 5 years ago

@Jaspreet23 I just did a dirty workaround, by not calling the register & de-register logic after every scenario. I am just calling that logic if a new feature is executed. You can do that in the StartingSteps Also for iOS to not re-install for each sceanrio, just add the capability "noReset":true in the test feed json file