search

getgauge / gauge-python

Python language runner for Gauge
MIT License
91 stars 38 forks source link

Multi-threading for python #121

Open iallen7 opened 5 years ago

iallen7 commented 5 years ago

Currently parallel execution using multi-threading for python is unsupported. Would like to see this supported in python as outlined in the 'parallel execution using threads' part of documentation:

capture

pmcphailWAT commented 5 years ago

This feature would be beneficial for testing embedded systems. Without the ability to use multi-threading for parallel execution we are severely limited to the number of tests we can run at a time. We run regression testing on 12+ control boards at a time from one computer and it looks like this is currently not possible if using python.

AkaAnto commented 5 years ago

I am facing the same issue, I have 240 test cases and it takes 6+ hours to run cause I can use parallel execution

NivedhaSenthil commented 5 years ago

Since python(cPython implementation) multithreading does not allow to run more than one thread at a time, we will not be able to utilise the full capacity of CPU cores. And thus not very suitable for computationally intensive processes. Refer for more discussions.

Currently gauge-python spawns multiple process based on CPU cores available to parallelise the execution.

@AkaAnto have you tried running specs in parallel ?

AkaAnto commented 5 years ago

I have with no luck, process spawns but I am facing a race condition. I am currently working on making all of my code thread safe to give it another try. I ll let you know how it goes

AkaAnto commented 5 years ago

I was able to run test in parallel @NivedhaSenthil. Race condition arise from time to time when using more than 3 process. This is a known bug for python 2.7. In a non related issue, any hook or way of knowing when all suites/test finished running ?

NivedhaSenthil commented 5 years ago

@after_suite hook should say that all the suite is executed and any action to be performed after suites completion can be done here. https://docs.gauge.org/latest/writing-specifications.html#id11