search

robotframework / PythonRemoteServer

Robot Framework remote server implemented with Python
Apache License 2.0
152 stars 83 forks source link

Asynchronous remote server #20

Open alans09 opened 10 years ago

alans09 commented 10 years ago

Hi,

it is possible to create asynchronous remote server? Using SocketServer.ThreadingMixIn or any other possible way?

It would be nice to have server that can accept more than one command in one time. It would definitely lowers the time to run keywords (eg. vmware connector to work with more than one machine at the time)

pekkaklarck commented 7 years ago

Sounds good to me. I don't have too much experience from asynchronous servers nor time to look at this in the foreseeable future, though. Anyone interested to create a PR?

pekkaklarck commented 7 years ago

I tested using ThreadingMixin and ForkingMixin simply inheriting them in addition to SimpleXMLRPCServer. Here are findings:

  1. Allows server to process multiple requests simultaneously as expected. Basically this means that one long lasting keyword doesn't block using other keywords.

  2. Somewhat surprisingly there were no performance benefits whatsoever when running tests with quick keywords in parallel. My guess is that the overhead of threads/processes is too big compared to benefits when keywords are run really fast.

  3. The current code for intercepting stdout/stderr isn't threadsafe. Currently we just do sys.stdout = StringIO() and later sys.stdout.getvalue() and sys.stdout = sys.__stdout__, but that doesn't work if multiple threads are changing sys.stdout. Results included AttributeError when using sys.stdout.getvalue() after another thread had already restored sys.stdout and also segfault with error Fatal Python error: GC object already tracked. We somehow needed to have own sys.stdout for each thread.

pekkaklarck commented 7 years ago

It seems that the thread safety problem could be fixed by using slightly modified version of the answers to this SO question: http://stackoverflow.com/questions/3029816/how-do-i-get-a-thread-safe-print-in-python-2-6

mohan-barathi commented 5 years ago

Hi @pekkaklarck ...! I have implemented this a couple of years back.

To overcome the thread safety problem, I implemented a mechanism in the robotframework library, where I keep track of stdout and stderr of each and every thread.

Also, I created separate output and logger objects, so that a single process can produce multiple output logs [and] reports in the same time, when multiple invocations come from different threads.

This also includes an implementation, where I authenticate the callee using IP Hence, this also handles the issue-51

I can give a try to port that implementation to the latest robotframework library and PythonRemoteServer, if you are still interested..!

Best Regards 😃

DanielPenning commented 2 years ago

Hi @mohan-barathi This sounds really interesting to me! Your comment dates a few years, are you by any chance still able to give it a try? If not, can you briefly lay out your thoughts about authentication?

SX91 commented 1 year ago

Hello! I've made a (very) simple alternative remote library implementation with async support via aiohttp and aiohttp_xmlrpc: https://github.com/SX91/aioroboremote It lacks proper (any) docs and some useful features of this library, yet supports basic type annotations and get_keyword_types. To get started you could follow the example in repo.

It's using the same stream interception approach, thus it's not multiclient safe.