Polling is sub-optimal

[leamas]

The async read implementation in boils down to polling. This is sub-optimal, and probably not acceptable in an "official" API. We need to somehow await that the file descriptor is ready.

[pylover]

Ye i agree with you, how about using epoll, and or select ?

[leamas]

Quite new inte async world. My recipe so far

• Create a async.Queue
• Add a reader func to the main loop (add_reader()). Called by nain loop when the fd could be read. and puts data in the queue.
• From here, just a ordinary async consumer which reads from the queue.

[leamas]

Here is proposed async interface to the current API. It's probably too simplistic, but the test code in the file header sort of works.

Note that I have also pushed some fixes to the underlying client.py.

async_client.py.zip

[pylover]

How the main loop gets aware of when fd is ready?

As i know, we have two options:

1. Calling the unix select without timeout, (The same as the f.read). this will be block the current thread, so i have to call it on another thread using python's asyncio blocking call mechanisms, which uses another thread.

2. Calling the select function with timeout, so its the same as the long polling you already mentioned in this issue.

So, is it ok to use another thread for feeding the queue, or you are thinking about another way which i don't know already?

[leamas]

But, there is also option 3, in my zipped async_client.py above. It basically boils down to loop.add_reader(fd, read_func). After this call, the main loop will cal read_func() as soon as there is data. My code is really just that.

I sort of like this, partly because it's very similar to how "extra" input sources are handled in GUI loops like GTK./Gkib. A proved pattern, that is.l

EDIT: TBH, I think this approach is better than 1) and 2) above. Adding yet another select within the select() context created by the main loop might create all sorts of problem, right?

EDIT2: It's probably not even feasible. The file descriptors are global...

[pylover]

Wow, interesting, i didn't know that. my bad. So, there is no problem,

Thanks in advance.

[leamas]

i didn't know that. my bad.

No reason to apologize! If I was to apologize for everything i don't know about python... we don't have time for that ;)

I have actually learnt a lot by looking at your code. Have implemented context managers in the base llibrary. So far, I don't think there are any breaking changes.

[pylover]

You already wrote the all of the parts, i just need to merge and test it. thanks

[leamas]

I found a bug in that code. close() should not close the connection, it's not created in that context. Leads to nasty crashes.

[pylover]

yes, some modifications are needed. but the structure is saving my time.I'm still wondering why i didn't know about add_reader function.