search

google / mobly

E2E test framework for tests with complex environment requirements.
https://github.com/google/mobly
Apache License 2.0
634 stars 177 forks source link

Implement an option to skip the remaining test suites, if a test fails during execution of suite runner. #673

Closed santhosh12992 closed 4 years ago

santhosh12992 commented 4 years ago

Hi,

If I am running multiple test suites using mobly suite runner, if a test suite fails, we need an option to abort or skip the remaining test suites.

Essentially, it means, we are following a sequence in the test execution and each test is a pre -requisite of following test suite.

Thanks. -Santhoshkumar M

xpconanfan commented 4 years ago

You can already do this with asserts.abort_all

On Sun, Jun 21, 2020, 12:23 AM Santhoshkumar Muralikrishnan < notifications@github.com> wrote:

Hi,

If I am running multiple test suites using mobly suite runner, if a test suite fails, we need an option to abort or skip the remaining test suites.

Essentially, it means, we are following a sequence in the test execution and each test is a pre -requisite of following test suite.

Thanks. -Santhoshkumar M

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/mobly/issues/673, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARDNZLYFQMORIHK5UQCKO3RXTO77ANCNFSM4ODOKRVA .

santhosh12992 commented 4 years ago

Just a guess,

should we define 'on_fail(record)' [which is under mobly.base_test module] with asserts.abort_all in the suite_runner calling class?

xpconanfan commented 4 years ago

Not sure what you mean. Please clarify

On Sun, Jun 21, 2020, 5:15 PM Santhoshkumar Muralikrishnan < notifications@github.com> wrote:

Just a guess,

should we define 'on_fail(record)' [which is under mobly.base_test module] with asserts.abort_all in the suite_runner calling class?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/mobly/issues/673#issuecomment-647102193, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARDNZIFTFNAE25XRDFPYUDRXXFUNANCNFSM4ODOKRVA .

santhosh12992 commented 4 years ago

The goal is abort all the tests using the api that you have mentioned, only when there is a failed test case across all test suite that is being run by suite runner.

Now the question is where to implement this, "asserts.abort_all".

xpconanfan commented 4 years ago

just call it in on_fail?

On Mon, Jun 22, 2020 at 4:14 PM Santhoshkumar Muralikrishnan < notifications@github.com> wrote:

The goal is abort all the tests using the api that you have mentioned, only when there is a failed test case across all test suite that is being run by suite runner.

Now the question is where to implement this, "asserts.abort_all".

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/mobly/issues/673#issuecomment-647360520, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARDNZMFMX5YMLFGFG4WHNLRX4HFLANCNFSM4ODOKRVA .

santhosh12992 commented 4 years ago

Thanks.