Hybrid Execution - Allow a test suite with manual and automated tests to be executed

[navaneeth]

Why?

[twchad]

The original ideas from a couple years back:

Makes test authoring and troubleshooting easier, as manual steps serve as debug points of sorts.

Hybrid tests also should speed up exploratory testing for testers, allowing them to speed through parts they aren't currently trying to test.

Lastly, hybrid tests let people gradually automate a manual test.

In practice, I'm not sure what people would do...

On Jan 11, 2015, at 9:33 AM, Navaneeth.K.N notifications@github.com wrote:

Why?

\ Reply to this email directly or view it on GitHub.

[navaneeth]

@twchad I think it is hardly used even in Twist. So I am not sure the benefit of bringing this into gauge.

And to support this, it needs an editor and gauge is all disconnected and not tied to the editor. Which makes it difficult to implement.

[prateekbaheti]

@navaneeth We could use a simple ui library in golang for implementation. It just needs simple UI for Passed, Failed and Error message for hybrid execution.

Some people do write hybrid scenarios when automation is difficult like for native interactions on browser. It helps to have specs written down in that case to ensure testing is done and reports are generated.

[navaneeth]

How do you turn this off when running on a CI? A flag to do that? Also, look at the compilation requirements for that UI library on Windows. It needs MinGW & cgo to perform the compilation. Again adding complexity to the build process and with this you no longer can cross-compile.

I'd say, talk to few more people about this before deciding. See if this will be really useful for a tester.

[jerger]

I like to add my vote for this feature. In fitNesse it was part of the USP to enable the domain people to write their own idea of test (I realized a hacky extension to support (such) manual test steps in fitNesse in order to provide a fast feedback while doing acceptance tests, and yes, in case of CI I disabled these manual steps).

After having such manual steps described,

1. domain people can gain fast experience, whether their test idea works (manually),
2. a dev can do the automation later on.

It's great to be able to involve domain people & it's also great to be able to manage the automation kappa & budget decoupled.

In terms of technical realization I would propose a light wight integration. Is there already a hook, where we can anchor such an manual and optional UI-test-executor plugin? Is there a way to tunnel an option thru the CLI down the test-executor? (Bdw, can I write plugins also in clojure / java also or do I've to write go?)

[mahendrakariya]

Thanks a lot for your opinion!

We haven't yet completely thought about how to implement this feature in Gauge. But what we know for sure is that this will require a change in Gauge core.

Is there already a hook, where we can anchor such an manual and optional UI-test-executor plugin?

Unfortunately, there is no hook available right now.

Is there a way to tunnel an option thru the CLI down the test-executor?

Gauge core orchestrates the actual execution of steps. We need to make relevant changes to core to pause the execution. Unless this is done, Gauge won't recognize any options passed through CLI.

(Bdw, can I write plugins also in clojure / java also or do I've to write go?)

Gauge plugins can be written in any language. In fact, only the core is in golang. All other plugins are in different languages. For instance, plugins like gauge-java, gauge-maven, gauge IntelliJ are written in Java. While plugins like gauge-csharp and VS are written in C#. There are plugins even in ruby, python, and javascript.

If you wish to write a Gauge plugin, let us know your idea and we will be more than happy to help you out in any way we can!

[jerger]

Thanks for your comment. In did I'm interested in writing a plugin - but first of all, I will need a clojure-language-runner issue 479.

So if you know others, having the same idea, I will be interested in getting in contact :-)

Bdw. your gitbook expandable-chapter-plugin is maintained by us - glad that you find this useful gitbook-plugin-expandable-chapters :-)

[mahendrakariya]

Thanks a lot for the gitbook-plugin-expandable-chapters!

We would welcome any contributions in terms of a new language runner! You can use other language runners like Java or C# for reference.

Just FYI, Clojure runner is not required to create a Gauge plugin. You can create a plugin in Clojure, even though Gauge doesn't have Clojure support. You can refer the docs for different types of Gauge plugins.

[jerger]

That's not a technical dependency - I know. I want use/test gauge to automate our dev-ops-system acceptance tests dda-pallet.

Dda-pallet is written in clojure, so we will need this clojure-language-runner upfront ...

[sriv]

Closing as stale. Please reopen if this is still valid.