search

PMExtra / sentry-auth-ldap

A Sentry extension to add an LDAP server as an authention source.
Apache License 2.0
25 stars 6 forks source link

AUTH_LDAP_USER_ATTR_MAP Mail mapping now working #14

Closed pavel-z1 closed 10 months ago

pavel-z1 commented 10 months ago

Map attribute of LDAP user account is not mapping

sentry on-premise(docker) 23.11.1 (also tested on 23.6.2) sentry-auth-ldap 23.6.1 FreeIPA as LDAP

During first ldap user authentication receive error:

07:06:39 [WARNING] django_auth_ldap: NotNullViolation('null value in column "email" of relation "sentry_useremail" violates not-null constraint\nDETAIL:  Failing row contains (20, null, G7knM8BqlW8j0iq9F25123456BrDZHRJR, 2023-11-24 07:06:39.472093+00, f, 8).\n')
SQL: INSERT INTO "sentry_useremail" ("user_id", "email", "validation_hash", "date_hash_added", "is_verified") VALUES (%s, %s, %s, %s, %s) RETURNING "sentry_useremail"."id" while authenticating testuser
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
sentry.models.useremail.UserEmail.DoesNotExist: UserEmail matching query does not exist.

LDAP user has external email address that stored in the mail attribute. ldapsearch return this mail attribute. But during Sentry auth mail attribute is not map to the new Sentry user.

my sentry.conf.py:

#############
# LDAP auth #
#############

import ldap
from django_auth_ldap.config import LDAPSearch, GroupOfUniqueNamesType

AUTH_LDAP_SERVER_URI = 'ldap://ipa.domain.local:389'

AUTH_LDAP_BIND_DB = 'uid=svc-sentry,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local'
AUTH_LDAP_BIND_PASSWORD = ''

AUTH_LDAP_USER_SEARCH = LDAPSearch(
    'cn=users,cn=accounts,dc=ipa,dc=domain,dc=local',
    ldap.SCOPE_SUBTREE, '(&(uid=%(user)s)(objectClass=posixAccount))',
)

AUTH_LDAP_CONNECTION_OPTIONS = {
    ldap.OPT_DEBUG_LEVEL: 1,
    ldap.OPT_REFERRALS: 0,
}

AUTH_LDAP_GROUP_SEARCH = LDAPSearch(
    "cn=groups,dc=ipa,dc=domain,dc=local", ldap.SCOPE_SUBTREE, "(objectClass=groupOfNames)"
)

AUTH_LDAP_USER_ATTR_MAP = {
    'name': 'cn',
    'email': 'mail',
}

AUTH_LDAP_FIND_GROUP_PERMS = False
AUTH_LDAP_CACHE_GROUPS = True
AUTH_LDAP_GROUP_CACHE_TIMEOUT = 3600

AUTH_LDAP_GROUP_TYPE = GroupOfUniqueNamesType()
AUTH_LDAP_REQUIRE_GROUP = None
AUTH_LDAP_DENY_GROUP = None

AUTH_LDAP_DEFAULT_SENTRY_ORGANIZATION = u'Sentry'
AUTH_LDAP_SENTRY_ORGANIZATION_ROLE_TYPE = 'member'
AUTH_LDAP_SENTRY_ORGANIZATION_GLOBAL_ACCESS = True
AUTH_LDAP_SENTRY_SUBSCRIBE_BY_DEFAULT = False

AUTH_LDAP_MAIL_VERIFIED = True

AUTHENTICATION_BACKENDS = AUTHENTICATION_BACKENDS + (
    'sentry_auth_ldap.backend.SentryLdapBackend',
)

# Optional logging for diagnostics.
LOGGING['disable_existing_loggers'] = False
import logging
logger = logging.getLogger('django_auth_ldap')
logger.setLevel(logging.DEBUG)

Full log:

07:06:39 [DEBUG] django_auth_ldap: Binding as
07:06:39 [DEBUG] django_auth_ldap: Invoking search_s('cn=users,cn=accounts,dc=ipa,dc=domain,dc=local', 2, '(&(uid=testuser)(objectClass=posixAccount))')
07:06:39 [DEBUG] django_auth_ldap: search_s('cn=users,cn=accounts,dc=ipa,dc=domain,dc=local', 2, '(&(uid=%(user)s)(objectClass=posixAccount))') returned 1 objects: uid=testuser,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local
07:06:39 [DEBUG] django_auth_ldap: Binding as uid=testuser,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local
07:06:39 [WARNING] django_auth_ldap: NotNullViolation('null value in column "email" of relation "sentry_useremail" violates not-null constraint\nDETAIL:  Failing row contains (20, null, G7knM8BqlW8j0iq12345JN3BrDZHRJR, 2023-11-24 07:06:39.472093+00, f, 8).\n')
SQL: INSERT INTO "sentry_useremail" ("user_id", "email", "validation_hash", "date_hash_added", "is_verified") VALUES (%s, %s, %s, %s, %s) RETURNING "sentry_useremail"."id" while authenticating testuser
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
sentry.models.useremail.UserEmail.DoesNotExist: UserEmail matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
    return func(self, sql, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/base.py", line 86, in execute
    return self.cursor.execute(sql, clean_bad_params(params))
psycopg2.errors.NotNullViolation: null value in column "email" of relation "sentry_useremail" violates not-null constraint
DETAIL:  Failing row contains (20, null, G7knM8BqlW8j0iq9F25PJJN3BrDZHRJR, 2023-11-24 07:06:39.472093+00, f, 8).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 77, in inner
    raise_the_exception(self.db, e)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 75, in inner
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 18, in inner
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 95, in inner
    raise exc_info[0](msg).with_traceback(exc_info[2])
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
    return func(self, sql, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/base.py", line 86, in execute
    return self.cursor.execute(sql, clean_bad_params(params))
psycopg2.errors.NotNullViolation: NotNullViolation('null value in column "email" of relation "sentry_useremail" violates not-null constraint\nDETAIL:  Failing row contains (20, null, G7knM8BqlW1234565PJJN3BrDZHRJR, 2023-11-24 07:06:39.472093+00, f, 8).\n')
SQL: INSERT INTO "sentry_useremail" ("user_id", "email", "validation_hash", "date_hash_added", "is_verified") VALUES (%s, %s, %s, %s, %s) RETURNING "sentry_useremail"."id"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/../sentry_sdk/integrations/django/views.py", line 84, in sentry_wrapped_callback
    return callback(request, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/silo/base.py", line 160, in override
    return original_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/web/frontend/base.py", line 390, in dispatch
    return self.handle(request, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/web/frontend/auth_organization_login.py", line 87, in handle
    response = self.handle_basic_auth(request, organization=organization)
  File "/usr/local/lib/python3.8/site-packages/sentry/web/frontend/auth_login.py", line 670, in handle_basic_auth
    elif login_form.is_valid():
  File "/usr/local/lib/python3.8/site-packages/django/forms/forms.py", line 175, in is_valid
    return self.is_bound and not self.errors
  File "/usr/local/lib/python3.8/site-packages/django/forms/forms.py", line 170, in errors
    self.full_clean()
  File "/usr/local/lib/python3.8/site-packages/django/forms/forms.py", line 373, in full_clean
    self._clean_form()
  File "/usr/local/lib/python3.8/site-packages/django/forms/forms.py", line 400, in _clean_form
    cleaned_data = self.clean()
  File "/usr/local/lib/python3.8/site-packages/sentry/web/forms/accounts.py", line 133, in clean
    self.user_cache = authenticate(username=username, password=password)
  File "/usr/local/lib/python3.8/site-packages/django/views/decorators/debug.py", line 42, in sensitive_variables_wrapper
    return func(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/__init__.py", line 76, in authenticate
    user = backend.authenticate(request, **credentials)
  File "/usr/local/lib/python3.8/site-packages/django_auth_ldap/backend.py", line 142, in authenticate
    user = self.authenticate_ldap_user(ldap_user, password)
  File "/usr/local/lib/python3.8/site-packages/django_auth_ldap/backend.py", line 200, in authenticate_ldap_user
    return ldap_user.authenticate(password)
  File "/usr/local/lib/python3.8/site-packages/django_auth_ldap/backend.py", line 344, in authenticate
    self._get_or_create_user()
  File "/usr/local/lib/python3.8/site-packages/django_auth_ldap/backend.py", line 597, in _get_or_create_user
    self._user, built = self.backend.get_or_build_user(username, self)
  File "/usr/local/lib/python3.8/site-packages/sentry_auth_ldap/backend.py", line 75, in get_or_build_user
    UserEmail.objects.update_or_create(defaults=defaults, user=user, email=mail)
  File "/usr/local/lib/python3.8/site-packages/sentry/silo/base.py", line 160, in override
    return original_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/silo/base.py", line 160, in override
    return original_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 608, in update_or_create
    obj, created = self.select_for_update().get_or_create(defaults, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/silo/base.py", line 160, in override
    return original_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
  File "/usr/local/lib/python3.8/site-packages/sentry/silo/base.py", line 160, in override
    return original_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/models/outboxes.py", line 258, in save
    super().save(*args, **kwds)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.8/site-packages/sentry/silo/base.py", line 160, in override
    return original_method(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/sentry/../sentry_sdk/integrations/django/__init__.py", line 641, in execute
    return real_execute(self, sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 77, in inner
    raise_the_exception(self.db, e)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 75, in inner
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 18, in inner
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 95, in inner
    raise exc_info[0](msg).with_traceback(exc_info[2])
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
    return func(self, sql, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sentry/db/postgres/base.py", line 86, in execute
    return self.cursor.execute(sql, clean_bad_params(params))
django.db.utils.IntegrityError: NotNullViolation('null value in column "email" of relation "sentry_useremail" violates not-null constraint\nDETAIL:  Failing row contains (20, null, G7knM8Bql123456PJJN3BrDZHRJR, 2023-11-24 07:06:39.472093+00, f, 8).\n')
SQL: INSERT INTO "sentry_useremail" ("user_id", "email", "validation_hash", "date_hash_added", "is_verified") VALUES (%s, %s, %s, %s, %s) RETURNING "sentry_useremail"."id"
07:06:39 [ERROR] django.request: Internal Server Error: /auth/login/sentry/ (status_code=500 request=<WSGIRequest: POST '/auth/login/sentry/'>)
PMExtra commented 10 months ago

I think there may be a blank email field in your LDAP entity.

Please note that an LDAP field is like an array and you can set multiple mail fields for an entity. If there is a null in it, the phenomenon you describe will occur.

If you confirm that this is indeed the cause and you cannot correct the data, you can try to add if mail: before line 75:

https://github.com/PMExtra/sentry-auth-ldap/blob/a98abf04c9b49b313e7d1283fafe86d2f1406ef2/sentry_auth_ldap/backend.py#L74-L75

pavel-z1 commented 10 months ago

Hi @PMExtra

ldapsearch return LDAP attibutes in this format for tested user:

mail: testuser@emaildomain.com
givenName: Test
sn: User
uid: testuser
cn: Test User
displayName: Test User
initials: TU
gecos: Test User
krbPrincipalName: testuser@IPA.DOMAIN.LOCAL

User contains only one email. It doesn't look like the LDAP mail attribute is in array format.

What can be done in this case to map LDAP mail attribute to sentry user?

PMExtra commented 10 months ago

Please try to do this and feedback me the outputs:

  1. Enter the sentry interactive shell:
docker exec -it sentry-web-1 sentry shell
  1. Import the LDAP backend and query the user attributes:
from django_auth_ldap.backend import LDAPBackend, _LDAPUser
user=_LDAPUser(LDAPBackend(), username='testuser')
print(user.attrs.get('mail'))
pavel-z1 commented 10 months ago

Strange results

Ldapsearch from cli:

ldapsearch -H ldap://$HOSTNAME -D 'uid=svc-sentry,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local' -W -b 'cn=users,cn=accounts,dc=ipa,dc=domain,dc=local' '(&(uid=tesstuser)(objectClass=posixAccount))' | grep mail
Enter LDAP Password:
mail: testuser@emaildomain.com

Sentry shell:


# docker exec -it sentry_onpremise-web-1 sentry shell
Python 3.8.18 (default, Nov 21 2023, 19:25:34)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django_auth_ldap.backend import LDAPBackend, _LDAPUser
>>> user=_LDAPUser(LDAPBackend(), username='testuser')
>>> print(user.attrs.get('mail'))
10:13:09 [DEBUG] django_auth_ldap: Binding as
10:13:09 [DEBUG] django_auth_ldap: Invoking search_s('cn=users,cn=accounts,dc=ipa,dc=domain,dc=local', 2, '(&(uid=testuser)(objectClass=posixAccount))')
10:13:09 [DEBUG] django_auth_ldap: search_s('cn=users,cn=accounts,dc=ipa,dc=domain,dc=local', 2, '(&(uid=%(user)s)(objectClass=posixAccount))') returned 1 objects: uid=testuser,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local
10:13:09 [DEBUG] django_auth_ldap: Invoking search_s('uid=testuser,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local', 0, '(objectClass=*)')
10:13:09 [DEBUG] django_auth_ldap: search_s('uid=testuser,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local', 0, '(objectClass=*)') returned 1 objects: uid=testuser,cn=users,cn=accounts,dc=ipa,dc=domain,dc=local
None
>>> print(user.attrs.get('givenName'))
['Test']
>>> print(user.attrs.get('krbPrincipalName'))
None
>>> print(user.attrs.get('displayName'))
['Test User']
>>>
PMExtra commented 10 months ago

You have not bound any identity, so you have no permission to access the attributes.

Please check AUTH_LDAP_BIND_DN and AUTH_LDAP_BIND_PASSWORD

pavel-z1 commented 10 months ago

Thank you @PMExtra

Typo AUTH_LDAP_BIND_DB instead of correct AUTH_LDAP_BIND_DN was the source of issue.