LDAP Authentication Matrix - can't login

[Winslow-AT]

Hello,

we're having problems getting the login with LDAP credentials (IBM Domino) on our HS working. Directory lookup and login via internal HS users works. We would be really thankful if you could help us out, we're kinda stuck now.

OS: Debian 9.9 Matrix: matrix-synapse-py3 - 0.99.3.2+stretch1 mxisd: mxisd - 1.4.3 Latest RestAuthProvider from here

HS: matrix.example.com IS: matrix.example.com

homeserver.yaml

default_identity_server: https://matrix.example.com

trusted_third_party_id_servers:
  - [HostIP]
  - matrix.example.com
  - localhost

password_providers:
  - module: "rest_auth_provider.RestAuthProvider"
    config:
      endpoint: "http://[HostIP]:8090"

mxisd.yaml

ldap:
  enabled: true
  connection:
    host: '[IBM Domino Server]'
    port: 389
    tls: false
    bindDn: 'cn=[bindUser]'
    bindPassword: '[bindPW]'
    baseDNs:
      - 'O=[Company]'
  attribute:
    uid:
      type: 'uid'
      value: 'dominounid'
    name: 'cn'
    threepid:
      email:
        - 'mail'

matrix:
  domain: 'matrix.example.com'

dns:
 overwrite:
   homeserver:
     client:
       - name: 'matrix.example.com'
         value: 'http://localhost:8008'

Apache Reverse Proxy vHost:

        ProxyPreserveHost               On
        ProxyPass                       /_matrix/identity                       http://[HostIP]:8090/_matrix/identity
        ProxyPassReverse                /_matrix/identity                       http://[HostIP]:8090/_matrix/identity
        ProxyPass                       /_matrix/client/r0/user_directory/      http://[HostIP]:8090/_matrix/client/r0/user_directory/
        ProxyPassReverse                /_matrix/client/r0/user_directory/      http://[HostIP]:8090/_matrix/client/r0/user_directory/
        ProxyPass                       /_matrix/client/r0/login                http://[HostIP]:8090/_matrix/client/r0/login
        ProxyPassReverse                /_matrix/client/r0/login                http://[HostIP]:8090/_matrix/client/r0/login
        ProxyPass                       /                                       https://[HostIP]:8448/
        ProxyPassReverse                /                                       https://[HostIP]:8448/

Login with "local" homeserver-User (root) via Riot-Desktop

Login with local users is working.

Log mxisd

[XNIO-1 task-19] INFO io.kamax.mxisd.auth.AuthManager - Proxy resolution: http://matrix.example.com/_matrix/client/r0/login to http://localhost:8008/_matrix/client/r0/login
[XNIO-1 task-20] INFO io.kamax.mxisd.auth.AuthManager - Login request is User ID type
[XNIO-1 task-20] INFO io.kamax.mxisd.auth.AuthManager - No User ID rewrite rules to apply
[XNIO-1 task-20] INFO io.kamax.mxisd.auth.AuthManager - Proxy resolution: http://matrix.example.com/_matrix/client/r0/login to http://localhost:8008/_matrix/client/r0/login
[XNIO-1 task-20] INFO io.kamax.mxisd.auth.AuthManager - http status = 200

Log Matrix

synapse.access.http.8008 - 233 - INFO - GET-967- - - 8008 - Received request: GET /_matrix/client/r0/login
synapse.access.http.8008 - 302 - INFO - GET-967- - - 8008 - {None} Processed request: 0.001sec/0.000sec (0.004sec, 0.000sec) (0.000sec/0.000sec/0) 41B 200 "GET /_matrix/client/r0/login HTTP/1.1" "mxisd/1.4.3" [0 dbevts]
synapse.access.http.8008 - 233 - INFO - GET-968- - - 8008 - Received request: GET /_matrix/client/r0/login
synapse.access.http.8008 - 302 - INFO - GET-968- - - 8008 - {None} Processed request: 0.001sec/0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/0) 41B 200 "GET /_matrix/client/r0/login HTTP/1.1" "mxisd/1.4.3" [0 dbevts]
synapse.handlers.typing - 89 - INFO - - Checking for typing timeouts
synapse.handlers.presence - 350 - INFO - - Handling presence timeouts
synapse.access.http.8008 - 233 - INFO - POST-969- - - 8008 - Received request: POST /_matrix/client/r0/login
synapse.rest.client.v1.login - 176 - INFO - POST-969- Got login request with identifier: {'user': 'root', 'type': 'm.id.user'}, medium: None, address: None, user: 'root'

Login with LDAP-User with email address via Riot-Desktop

Log mxisd

[XNIO-1 task-8] INFO io.kamax.mxisd.auth.AuthManager - Login request with medium 'email' and address '[validMail]'
[XNIO-1 task-8] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Performing LDAP lookup [validMail] of type email
[XNIO-1 task-8] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Query: (|(mail=[validMail]))
[XNIO-1 task-8] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Attributes: "dominounid"
[XNIO-1 task-8] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Base DN: O=[Company]
[XNIO-1 task-8] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Found possible match, DN: CN=Michael Hoessel,OU=Deutsche Berufskleider-Leasing,O=[company]
[XNIO-1 task-8] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - DN CN=[correctUser] is a valid match
[XNIO-1 task-8] INFO io.kamax.mxisd.lookup.strategy.RecursivePriorityLookupStrategy - Found 3PID mapping: {medium: 'email', address: '[validMail]', mxid: '@837A422D2D1931ABC1257D640023E6EF:matrix.example.com'}
[XNIO-1 task-8] INFO io.kamax.mxisd.auth.AuthManager - Proxy resolution: http://matrix.example.com/_matrix/client/r0/login to http://localhost:8008/_matrix/client/r0/login
[XNIO-1 task-8] INFO io.kamax.mxisd.auth.AuthManager - http status = 500

Log Matrix

synapse.access.http.8008 - 233 - INFO - POST-3976- - - 8008 - Received request: POST /_matrix/client/r0/login
synapse.rest.client.v1.login - 176 - INFO - POST-3976- Got login request with identifier: {'user': '837A422D2D1931ABC1257D640023E6EF', 'type': 'm.id.user'}, medium: None, address: None, user: None
synapse.storage._base - 401 - WARNING - - Starting db txn 'get_users_by_id_case_insensitive' from sentinel context
synapse.storage._base - 437 - WARNING - - Starting db connection from sentinel context: metrics will be lost
synapse.handlers.auth - 602 - WARNING - - Attempted to login as @837A422D2D1931ABC1257D640023E6EF:matrix.example.com but they do not exist
synapse.http.server - 112 - ERROR - - Failed handle request via 'LoginRestServlet': <XForwardedForRequest at 0x7f3334619e10 method='POST' uri='/_matrix/client/r0/login' clientproto='HTTP/1.1' site=8008>
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/server.py", line 81, in wrapped_request_handler
    yield h(self, request)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/server.py", line 316, in _async_render
    callback_return = yield callback(request, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/client/v1/login.py", line 149, in on_POST
    result = yield self._do_other_login(login_submission)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/client/v1/login.py", line 246, in _do_other_login
    login_submission,
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/handlers/auth.py", line 679, in validate_login
    qualified_user_id, password,
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/ldap_auth_provider.py", line 187, in check_password
    mail = attrs[self.ldap_attributes['mail']][0]
IndexError: list index out of range

Summary: Login gets redirected to mxisd Credentials are then posted to HS LDAP lookup is working Login with LDAP credentials/creation of users won't work - ???

Questions: Well, why doesn't it work? :D

We tried multiple configs, other attributes ... can't get it to work.

Thanks in advance for any help

[maxidorius]

Your problem is here:

  File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/ldap_auth_provider.py", line 187, in check_password
    mail = attrs[self.ldap_attributes['mail']][0]

Synapse is calling the LDAP password provider which I'm guessing is a left-over of an older configuration, probably in the conf.d folder? That password provider must not be used alongside mxisd.

The REST Password Provider is not used and is why it's failing.

[Winslow-AT]

Thanks for the fast and helpfull response!

You're right, there was a left-over and I fixed it, we're getting closer - now this is the output:

mxisd - Correct Password

[XNIO-1 task-13] INFO io.kamax.mxisd.auth.AuthManager - Login request with medium 'email' and address '[validMail]'
[XNIO-1 task-13] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Performing LDAP lookup [validMail] of type email
[XNIO-1 task-13] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Query: (|(mail=[validMail]))
[XNIO-1 task-13] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Attributes: "dominounid"
[XNIO-1 task-13] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Base DN: O=[Company]
[XNIO-1 task-13] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Found possible match, DN: [validUser]
[XNIO-1 task-13] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - DN [validUser] is a valid match
[XNIO-1 task-13] INFO io.kamax.mxisd.lookup.strategy.RecursivePriorityLookupStrategy - Found 3PID mapping: {medium: 'email', address: '[validMail]', mxid: '@837A422D2D1931ABC1257D640023E6EF:matrix.example.com'}
[XNIO-1 task-13] INFO io.kamax.mxisd.auth.AuthManager - Proxy resolution: http://matrix.example.com/_matrix/client/r0/login to http://localhost:8008/_matrix/client/r0/login
[XNIO-1 task-14] INFO io.kamax.mxisd.http.undertow.handler.auth.RestAuthHandler - Requested to check credentials for @837A422D2D1931ABC1257D640023E6EF:matrix.example.com
[XNIO-1 task-14] INFO io.kamax.mxisd.auth.AuthManager - Attempting authentication with store LdapAuthProvider
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Performing auth for @837A422D2D1931ABC1257D640023E6EF:matrix.example.com
[XNIO-1 task-14] DEBUG io.kamax.mxisd.backend.ldap.LdapAuthProvider - Query: (dominounid=837A422D2D1931ABC1257D640023E6EF)
[XNIO-1 task-14] DEBUG io.kamax.mxisd.backend.ldap.LdapAuthProvider - Attributes: ["mail","cn","dominounid"]
[XNIO-1 task-14] DEBUG io.kamax.mxisd.backend.ldap.LdapAuthProvider - Base DN: O=[Company]
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Checking possible match, DN: [validUser]
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Attempting authentication on LDAP for [validUser]
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Authentication successful for [validUser]
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - DN [validUser] is a valid match
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Processing 3PIDs for profile
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Processing 3PID type email
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - #011Attribute mail has 1 value(s)
[XNIO-1 task-14] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Found 1 3PIDs
[XNIO-1 task-14] INFO io.kamax.mxisd.auth.AuthManager - @837A422D2D1931ABC1257D640023E6EF:matrix.example.com was authenticated by LdapAuthProvider, publishing 3PID mappings, if any
[XNIO-1 task-14] INFO io.kamax.mxisd.auth.AuthManager - Processing io.kamax.matrix.ThreePid@f3c6f1a7 for @837A422D2D1931ABC1257D640023E6EF:matrix.example.com
[XNIO-1 task-14] INFO io.kamax.mxisd.invitation.InvitationManager - Looking up possible pending invites for email:[validMail]
[XNIO-1 task-13] INFO io.kamax.mxisd.auth.AuthManager - http status = 403

mxisd - Wrong password

[XNIO-1 task-16] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Performing LDAP lookup [validMail] of type email
[XNIO-1 task-16] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Query: (|(mail=[validMail]))
[XNIO-1 task-16] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Attributes: "dominounid"
[XNIO-1 task-16] DEBUG io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Base DN: O=[Company]
[XNIO-1 task-16] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - Found possible match, DN: [validUser]
[XNIO-1 task-16] INFO io.kamax.mxisd.backend.ldap.LdapThreePidProvider - DN [validUser] is a valid match
[XNIO-1 task-16] INFO io.kamax.mxisd.lookup.strategy.RecursivePriorityLookupStrategy - Found 3PID mapping: {medium: 'email', address: '[validMail]', mxid: '@837A422D2D1931ABC1257D640023E6EF:matrix.example.com'}
[XNIO-1 task-16] INFO io.kamax.mxisd.auth.AuthManager - Proxy resolution: http://matrix.example.com/_matrix/client/r0/login to http://localhost:8008/_matrix/client/r0/login
[XNIO-1 task-17] INFO io.kamax.mxisd.http.undertow.handler.auth.RestAuthHandler - Requested to check credentials for @837A422D2D1931ABC1257D640023E6EF:matrix.example.com
[XNIO-1 task-17] INFO io.kamax.mxisd.auth.AuthManager - Attempting authentication with store LdapAuthProvider
[XNIO-1 task-17] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Performing auth for @837A422D2D1931ABC1257D640023E6EF:matrix.example.com
[XNIO-1 task-17] DEBUG io.kamax.mxisd.backend.ldap.LdapAuthProvider - Query: (dominounid=837A422D2D1931ABC1257D640023E6EF)
[XNIO-1 task-17] DEBUG io.kamax.mxisd.backend.ldap.LdapAuthProvider - Attributes: ["mail","cn","dominounid"]
[XNIO-1 task-17] DEBUG io.kamax.mxisd.backend.ldap.LdapAuthProvider - Base DN: O=[Company]
[XNIO-1 task-17] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Checking possible match, DN: [validUser]
[XNIO-1 task-17] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Attempting authentication on LDAP for [validUser]
[XNIO-1 task-17] INFO io.kamax.mxisd.backend.ldap.LdapAuthProvider - Unable to bind using [validUser] because Failed, invalid credentials for [validUser]
[XNIO-1 task-16] INFO io.kamax.mxisd.auth.AuthManager - http status = 403

Synapse - Wrong password

synapse.access.http.8008 - 233 - INFO - POST-12- - - 8008 - Received request: POST /_matrix/client/r0/login
synapse.rest.client.v1.login - 176 - INFO - POST-12- Got login request with identifier: {'user': '837A422D2D1931ABC1257D640023E6EF', 'type': 'm.id.user'}, medium: None, address: None, user: None
synapse.handlers.auth - 602 - WARNING - POST-12- Attempted to login as @837A422D2D1931ABC1257D640023E6EF:matrix.example.com but they do not exist
synapse.handlers.auth - 602 - WARNING - POST-12- Attempted to login as @837A422D2D1931ABC1257D640023E6EF:matrix.example.com but they do not exist
synapse.http.server - 85 - INFO - POST-12- <XForwardedForRequest at 0x7fc0808a5550 method='POST' uri='/_matrix/client/r0/login' clientproto='HTTP/1.1' site=8008> SynapseError: 403 - Invalid password

Synapse - Correct password

synapse.access.http.8008 - 233 - INFO - POST-10- - - 8008 - Received request: POST /_matrix/client/r0/login
synapse.rest.client.v1.login - 176 - INFO - POST-10- Got login request with identifier: {'user': '837A422D2D1931ABC1257D640023E6EF', 'type': 'm.id.user'}, medium: None, address: None, user: None
synapse.handlers.auth - 602 - WARNING - POST-10- Attempted to login as @837A422D2D1931ABC1257D640023E6EF:matrix.example.com but they do not exist
synapse.http.server - 85 - INFO - POST-10- <XForwardedForRequest at 0x7fc0808a9080 method='POST' uri='/_matrix/client/r0/login' clientproto='HTTP/1.1' site=8008> SynapseError: 403 - Invalid password

My expectation is that the HS should create the user after the IS validated it. Besides that - in the case of a wrong password - is it intended to still call the HS even if the IS stated "invalid credentials"? Shouldn't it stop there?

[maxidorius]

My expectation is that the HS should create the user after the IS validated it.

Only if the user doesn't exist in synapse DB yet, but since you've tried to logging with a different password provider, I don't know what is the DB state. I've doubled checked with synapse 0.99.3 and 0.99.4 and mxisd 1.4.3 and it works fine on our end (we use Samba tho).

Check that no password is present in the synapse DB for the user you try to log in as. the table is users.

Besides that - in the case of a wrong password - is it intended to still call the HS even if the IS stated "invalid credentials"? Shouldn't it stop there?

The HS is not called again, it's just the completition of the previous request. mxisd is multi-threaded, so you need to check what thread each action relates to and when each was started.

[Winslow-AT]

Check that no password is present in the synapse DB for the user you try to log in as. the table is users.

Checked that, only 3 manually created user listed there.

If I create the user with the ID manually and set the password it works. Client -> IS -> Check LDAP with PW -> HS -> Login successfull

Without the manual user creation it fails.

# Enable registration for new users.
#
#enable_registration: false

I only enabled "registration_shared_secret" in synapse, do you need to enable something else there for the creation?

[Winslow-AT]

After further testing and debugging:

synapse.handlers.auth - 602 - WARNING - POST-2- Attempted to login as @837A422D2D1931ABC1257D640023E6EF:matrix.example.com but they do not exist
synapse.handlers.auth - 602 - WARNING - POST-2- Attempted to login as @837A422D2D1931ABC1257D640023E6EF:matrix.example.com but they do not exist
synapse.http.server - 85 - INFO - POST-2- <XForwardedForRequest at 0x7f9d905c7908 method='POST' uri='/_matrix/client/r0/login' clientproto='HTTP/1.1' site=8008> SynapseError: 403 - Invalid password

So... the user doesn't exist but the password is invalid... okay?

Also tried with AD instead of IBM Domino, pretty much the same thing.

[maxidorius]

@Winslow-AT Is there a Matrix ID at which I can contact you directly? You can send me an email too if you want to keep it private, see my github profile for a recipient address.

[maxidorius]

Issue was the localpoart in the Matrix IDs using illegal characters, upper case characters in this instance.