Test classes don't run in sequence

levank707 commented 3 years ago

TestNG Version

7.3.0

About Issue

test classes do not run in sequence for ex: I have 2 test classes, say, com.qa.tests.LoginTests and com.qa.tests.ProductsTest . in each class is two methods when I am running it on one device its running in sequence, but when I am running it in parallel it don't run in sequence. please see below .xml file.

Expected behavior.

com.qa.tests.LoginTests - 1 method com.qa.tests.LoginTests - 2 method com.qa.tests.ProductsTest - 1 method com.qa.tests.ProductsTest - 2 method

Actual behavior

com.qa.tests.LoginTests - 1 method com.qa.tests.ProductsTest - 1 method com.qa.tests.LoginTests - 2 method com.qa.tests.ProductsTest - 2method

Test case sample

<?xml version="1.0" encoding="UTF-8"?>

juherr commented 3 years ago

Please, try the latest TestNG version. If the issue still exists, please share a reproducible project.

levank707 commented 3 years ago

first of all , thanks for your response. sorry I had a mistake in description.. I am using latest version of TESTNG 7.3.0 here is one thing, when I am executing only "com.qa.tests.LoginTests" parallel on each device its running test in sequence . but when I added second test class "com.qa.tests.ProductsTest" its don't run in sequence. its running like, loginTest first method on each device , then ProductsTest first method on each device , and so... one more thing, without parallel execution tests are running in sequence. p.s sorry for my english xD

levank707 commented 3 years ago

Please, try the latest TestNG version. If the issue still exists, please share a reproducible project.

I log methods and here is result, its don't run i sequence

22:iOS:iPhone 11 Pro:com.qa.tests.LoginTests: ** STARTING TEST: invalidUserName **

23:Android:Pixel_3:com.qa.tests.LoginTests: ** STARTING TEST: invalidUserName **

23:Android:Pixel_3:com.qa.tests.ProductsTest: ** STARTING TEST: productName **

22:iOS:iPhone 11 Pro:com.qa.tests.ProductsTest: ** STARTING TEST: productName **

22:iOS:iPhone 11 Pro:com.qa.tests.LoginTests: ** STARTING TEST: invalidPassword **

23:Android:Pixel_3:com.qa.tests.LoginTests: ** STARTING TEST: invalidPassword **

I want to first was running LoginTests methods and after that ProductsTest methods, but, but , here is result :/

krmahadevan commented 3 years ago

@levank707 - We cannot do much with either screenshots or your test execution logs. You would need to do one of the following:

Create a sample Maven|Gradle project which uses TestNG 7.3.0 and can be used to reproduce the problem at hand (or)
Atleast share a test case example that can be used to reproduce the problem.
Include what is the expected output and the actual output.
Make sure that you dont use any other libraries such as Selenium|Appium etc., in the sample project (or) test case example, but just use System.out.println statements to simulate the problem.

Without the aforementioned details, this issue is likely to be closed out.

Thanks in advance for your understanding.

levank707 commented 3 years ago

@levank707 - We cannot do much with either screenshots or your test execution logs. You would need to do one of the following:

Create a sample Maven|Gradle project which uses TestNG 7.3.0 and can be used to reproduce the problem at hand (or)

Atleast share a test case example that can be used to reproduce the problem.

Include what is the expected output and the actual output.

Make sure that you dont use any other libraries such as Selenium|Appium etc., in the sample project (or) test case example, but just use System.out.println statements to simulate the problem.

Without the aforementioned details, this issue is likely to be closed out.

Thanks in advance for your understanding.

here is the project https://github.com/levank707/appiumcicd.git

levank707 commented 3 years ago

Please, try the latest TestNG version. If the issue still exists, please share a reproducible project.

https://github.com/levank707/appiumcicd.git

krmahadevan commented 3 years ago

@levank707 - this project which you shared has selenium dependencies. We won't be able to use it and debug. Please update the project such that it doesn't use any external dependencies ( just uses System.out.println statements )

levank707 commented 3 years ago

@levank707 - this project which you shared has selenium dependencies. We won't be able to use it and debug. Please update the project such that it doesn't use any external dependencies ( just uses System.out.println statements )

LoginTest {
@test(priority = 0) Login.firstMethod{ } @test(priority = 1) Login.secondMethod{ } }

ProductsTest{ @test(priority = 0) products.firstMethod{ } @test(priority = 1) products.secondMethod{ } }

output is Login.firstMethod product.firstMethod Login.secondMethod product. secondMethod

how avoid this? I don't want to run it by alphabet, this was reason why I set priority on each method.

juherr commented 3 years ago

@levank707 priority is shared in all the suite. What you want to use is dependsOnMethods.

levank707 commented 3 years ago

Guys thanks, your advices helped me to debug it correctly and to find a problem, the reason why I set priority on each method was, that I didn't want to run it by alphabet i want it to run in sequence so, right now I will set first character on each methods like A,B,C,D, etc. It would be ok ? or is there any other solution for it

juherr commented 3 years ago

You can sort methods with dependsOnMethods.

There is no strong rules on the default order and it may change between 2 versions. That's why it is not a good idea to force the order with an alphabet prefix.

levank707 commented 3 years ago

You can sort methods with dependsOnMethods.

There is no strong rules on the default order and it may change between 2 versions. That's why it is not a good idea to force the order with an alphabet prefix.

Thanks juherr, wish you all the best

testng-team / testng