Unable to set connect timeout

[magnuswatn]

Hi,

The "timeout" parameter on LDAPClient.connect() seems to only affect the LDAP bind process, and not the network connection. So if the network connection times out (aka. SYN timeout) Bonsai seems to hang forever.

Example code:

import bonsai

def main():
    cli = bonsai.LDAPClient("ldap://bonsai.test.watn.no")
    print("Connecting...")
    cli.connect(timeout=2.0)
    print("Connected!")
    cli.close()

if __name__ == "__main__":
    main()

(bonsai.test.watn.no drops packets on port 389, so the connection will end in SYN timeout).

This also affect LDAPS connections if the SSL handshake times out. I'm testing on Linux, Fedora 27, with OpenLDAP 2.4.45-4.fc27.

(otherwise; great module! Thanks for making it.)

[noirello]

Hi, that seems interesting. Could you give me some details how can I test this?

[magnuswatn]

Hi!

"bonsai.test.watn.no" should behave the same way for you as for me, so just running the example code should do the trick. That's the SYN timeout one, I can make a test for the SSL handshake timeout as well, if you wish.

But, I see now that I jumped the gun when I said "just hangs". The connection attempt fails after about two minutes:

>>> datetime.now().isoformat()
'2018-06-18T20:28:58.006610'
>>> main()
Connecting...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in main
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/ldapclient.py", line 555, in connect
    return LDAPConnection(self, is_async).open(timeout)
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/ldapconnection.py", line 101, in open
    return self._evaluate(super().open(), timeout)
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/ldapconnection.py", line 52, in _evaluate
    return self.get_result(msg_id, timeout)
bonsai.errors.ConnectionError: Can't contact LDAP server. (0xFFFF [-1])
>>> datetime.now().isoformat()
'2018-06-18T20:31:08.471086'
>>> 

This is a bit on the high side for my project, so it would be great to be able to adjust it.

[noirello]

Oh, that's cool. Should have checked the example, before I asked. Looked into it, and unfortunately there's no workaround for it in the current releases. :(

But it might be fixed in a future one. There is an LDAP_OPT_NETWORK_TIMEOUT option in OpenLDAP specifically for this. By setting that, the connection honours the timeout. Actually, this option would be the key for truly asynchronous connection build-ups (setting it to 0), but I had bad experience with it: on some OSes (Ubuntu, Mac, ...) the LDAP operations somehow polled the socket's status incorrectly, and tried to use it earlier than they should, while on other OSes (e.g. Fedora) it worked just fine. It's on my todo list to check this again in the near future.

[magnuswatn]

I have now configured bonsai.test.watn.no to drop the connection during the ssl handshake on ldaps, so you can try out the handshake timeout as well. It seems to hang forever (or at least > 5 minutes):

>>> datetime.now().isoformat()
'2018-06-18T23:01:08.816638'
>>> main()
Connecting...
^C^C^C^C^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in main
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/ldapclient.py", line 555, in connect
    return LDAPConnection(self, is_async).open(timeout)
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/ldapconnection.py", line 101, in open
    return self._evaluate(super().open(), timeout)
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/ldapconnection.py", line 52, in _evaluate
    return self.get_result(msg_id, timeout)
  File "/home/mhw/.local/share/virtualenvs/api-Zf5oDCEJ/lib/python3.6/site-packages/bonsai/errors.py", line 181, in _get_error
    def _get_error(code: int) -> type:
KeyboardInterrupt
>>> 
>>> datetime.now().isoformat()
'2018-06-18T23:06:53.123616'
>>> 

But yeah, sounds like LDAP_OPT_NETWORK_TIMEOUT is the right knob to adjust. Wonderful if you could take a look in the near future - "truly asynchronous" sounds pretty sweet :-)

[noirello]

Thanks, I tried, and it seems like that the SSL connection is even beyond reach for OpenLDAP. It still hangs after setting LDAP_OPT_NETWORK_TIMEOUT. My guess that the timeout param is not pass down to the SSL connection.

You can use bonsai.set_debug(True, 1) to see what step is actually stuck.

[noirello]

Ok, I've looked into it. I set the LDAP_OPT_NETWORK_TIMEOUT to 0 and the LDAP_OPT_CONNECT_ASYNC flag on before opening a connection, rebuilt the module, then ran the tests on the last 3 Ubuntu LTS releases and on an older Fedora. The results are:

• 14.04-trusty: libldap 2.4.31, all connection related tests failed.
• Fedora 22: libldap 2.4.40, all connection related tests failed.
• 16.04-xenial: libldap 2.4.42, only TLS related tests failed.
• 18.04-bionic: libldap 2.4.45, all tests were ok.

Then I found this in OpenLDAP 2.4.41's Change Log:

Fixed libldap async connections (ITS#8090)

Oddly, there's no mention about a TLS fix in the change logs, but it has been fixed since 2.4.43, or at least that's what I've experienced.

So the next step will be to add a version test macro in the source and set these options when newer libldap is used to build the extension. That will grant us async connection built-up, and solve your problem as a non-negligible "side-effect". :)

Or that'd be the next step, but I also ran the test on a Mac OS, and it failed miserably, regardless of the fact that the extension was built with a very new libldap 2.4.46. I'll investigate this a little bit more, before making the changes.

[magnuswatn]

Sounds very good! :+1:

[noirello]

The new release is out and it contains the fix for libldap 2.4.43+ on Linux.

[magnuswatn]

Thanks!