Asynchronous operations without asyncio

[hkoof]

I've been trying to do an asynchronous LDAP-search with PyLDAP. But for the app I would like to make, there is already an eventloop (which is not based python3.4's asyncio). I could not see a way how to do asynchronous operations without having an asyncio-eventloop.

While I think it is a very nice idea to support asyncio's yield from pattern from a ldap-module, I would consider it a "bug" if it is not possible to do asynchronous operations without it.

[edit] Put quotes around "bug". [/edit]

[noirello]

Well, I have to say, that this is a "bug" as you refer it. So far I didn't look into other asynchronous library than asyncio. If you tell me what other library would like to use, I will look into it in the future.

[hkoof]

I did not want to suggest supporting other eventloops too as a solution for this "bug" (I do agree with the quotes here).

Since iosync is part of python now, I think it is very nice to have a python3-ldap-module support it. That is even how I found PyLDAP... But I do think depending on an asyncio-eventloop instance being present in a program is a disadvantage of such a lib (well, at least for me it is, at this point in time). Especially because of the nature of event-loops: it is not possible to run two event-loop-instances.

So I'd like to refrase this issue as a wishlist item: I'd like to suggest implementing support for getting an opaque handle returned for async ops (in addition to asyncio support). Just like the C-lib, ldap for python2 and ldap3 do. That way async ops can be used in other event loops or even without any eventloop.

[noirello]

The fact is that the C-based LDAPConnection and LDAPEntry already use only asynchronous LDAP C functions, and all of the LDAP operations return a poll-able message ID instead of the actual result. But this functionality is covered with Python code on the surface (and not advertised at all).

In the past weeks I started to make some progress about being compatible with other async library, not just with asyncio.

• I removed the burnt private Python attribute calls from the C code (changed them to properties), which makes easier to inherit from the classes.
• I added a fileno method to LDAPConnection that returns the underlying socket descriptor, thus select based event-loops can be used.
• Because the initialisation, TLS build-up and binding are blocking procedures, they take place in a separate thread, but to be able to use select on these operations I use internal socket pairs to send messages from one to the other when the thread functions are finished. During the connection build-up, one of the socket's descriptor is returned by the fileno method and can be used for select.

I also added a new method to the LDAPClient (set_poll_func) that can be used for changed the default asyncio-based poll function to some custom function. I've created a gevent-based poll function for testing purposes that seems to be working well with gevent's event loop, but I am not sure that this "setting a different poll function" approach is the best, maybe a simple inheritance would be better (so that could be changed before a new release).

All of the modifications are uploaded on the dev branch.

[noirello]

New release on the master branch, added Gevent and Tornado support. Created separate connection class for each library, that can be change with LDAPClient.set_async_connection_class() method.

New classes that support for other asynchronous library can be written based on these classes.