parallelize appium builds

[tomholub]

tests are starting to take an hour to run. Please try setting tests to run concurrently - up to 2 tests at a time.

[fcvakintos]

You mean to create queue on semaphore?

[tomholub]

I mean when I run npm run-script only.test.all on CI, it will start two simulators to run tests in parallel / get done faster. Could add a sysarg like npm run-script only.test.all --parallel. When present, appium will start two simulators instead of one, and test in parallel.

[fcvakintos]

I know how to run same tests but for multiple devices, but don't know how to do that if we want to split them. Need to google

[tomholub]

For example on browser extension we parallelize most tests to run 10 in parallel (except for the particularly flaky ones which we run sequentially). For iOS, we can start with 2.

[fcvakintos]

mobile part is much more complicated if compare with browser

[tomholub]

I know

[fcvakintos]

@tomholub I have idea, we could use semaphore blocks to run separate list of tests so in that case they will be devided and faster. What do you think based on your exp with this CI system?

[tomholub]

We could always do that, but we should first fully utilize a single machine before increasing amount of machines.

[fcvakintos]

Parallels are possible if start multiple appium server instances for different ports, different simulators and somehow need to devide tests... So many points to think

[tomholub]

does appium not already provide any mechanism for this?

[fcvakintos]

no, what I saw is just running same tests on diff devices but it's not what we need

[tomholub]

ok. Let's leave this for later

[ioanmo226]

@sosnovsky I think this issue seems outdated as we already run appium tests in multiple jobs. What do you think?

[sosnovsky]

Yes, we can close this issue as tests are already running in parallel.