search

project8 / dripline-python

python implementation of project8/dripline
Other
2 stars 0 forks source link

send_request should make use of the async backend #3

Closed guiguem closed 5 years ago

guiguem commented 7 years ago

From @laroque on September 18, 2015 23:46

WARNING!... this isn't actually a starter project, it is kind of advanced.

Currently the sending a request involves using threading and a new connection to send a message and get a reply. This feels super dirty. I'd like to do something like:

timeout_time = datetime.datetime.now() + datetime.timedelta(seconds=timeout_in_seconds)
correlation_id = uuid.uuid4()
self.send_message(<your request message stuff>, correlation_id=correlation_id)
while self._results[correlation_id] is None and datetime.datetime.now < timeout_time:
     #some yielding block that lets the service process any new messages

... but I haven't got a solution for the yielding code block

should just send a request and then watch some local dictionary of responses for one with a matched correlation_id... need to have solutions to two issues: 1) still need to support timeout for getting the reply 2) need to make sure data-structures don't grow without bound if either replies collect but never get sent anywhere, or records of replies awaiting with reply do not grow similarly

Copied from original issue: project8/dripline#135

guiguem commented 7 years ago

From @laroque on September 19, 2015 5:16

ugh... why do I keep doing this to myself, the send_request method of service is blocking, which means that it cannot then handle_reply because it is still trying to send_request.... python 3's yield from syntax may offer a nice solution to this, I'll play with pika 0.10 and python 3.4 at some point, but for now I should give up

laroque commented 6 years ago

As a note, as of pika 0.11.0, there now exists an AsyncConnection adapter natively in pika, which uses asyncio's event loop. It may be possible to change the adapter with few other changes to the service class and achieve asynchronous message handling. Since we haven't worked with 0.10.0 we should be sure to jump to 0.11.0 and leverage this feature whenever we decide to jump into this.

wcpettus commented 5 years ago

We are migrating away from pika and will solve this with pybind'ing dripline-cpp