pennersr / django-allauth

Integrated set of Django applications addressing authentication, registration, account management as well as 3rd party (social) account authentication.
https://allauth.org
MIT License
9.12k stars 2.98k forks source link

Can't log in MultipleObjectsReturned at /accounts/login/ #3019

Closed msapiro closed 11 months ago

msapiro commented 2 years ago

A user signs up for an account with a mixed case email address like User@example.com and confirms the address. This creates a user with primary email User@example.com. The user subsequently logs out and then logs in again with address user@example.com. This creates a secondary email address for the user with the lower-cased address. Now once the user logs out a subsequent login attempt fails with this traceback:

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper
    return view(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 146, in dispatch
    return super(LoginView, self).dispatch(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 74, in dispatch
    response = super(RedirectAuthenticatedUserMixin, self).dispatch(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 97, in dispatch
    return handler(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 102, in post
    response = self.form_valid(form)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 159, in form_valid
    return form.login(self.request, redirect_url=success_url)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/forms.py", line 196, in login
    ret = perform_login(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 174, in perform_login
    if not _has_verified_for_login(user, email):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 134, in _has_verified_for_login
    emailaddress = EmailAddress.objects.get_for_user(user, email)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/managers.py", line 54, in get_for_user
    ret = self.get(user=user, email__iexact=email)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py", line 419, in get
    raise self.model.MultipleObjectsReturned(

Exception Type: MultipleObjectsReturned at /accounts/login/
Exception Value: get() returned more than one EmailAddress -- it returned 2!

Note that this does not occur if the initial creation is with a lower-cased address. In that case, a subsequent login with a mixed case address does not create a secondary address record.

Also note that once the secondary email address is created, the user can't log in with either email address, but is able to log in with the user name.

I understand this is tricky because local parts can be case sensitive, but one way to avoid this is on initial sign-up, lower-case the email address. Or perhaps catch the MultipleObjectsReturned exception and ignore it if the addresses belong to the same user.

This is with Django==3.0.14 and django-allauth==0.47.0 and also with Django==3.2.11 and django-allauth==0.47.0

pennersr commented 1 year ago

The user subsequently logs out and then logs in again with address user@example.com This creates a secondary email address for the user

I have trouble relating the above statement to the code. Logging in does not / should not create a secondary email address. Can you try capturing what you are experiencing in a test case?

msapiro commented 1 year ago

With Django==3.2.16 and django-allauth==0.51.0 and this in urlpatterns in urls.py

    url(r'^accounts/', include('allauth.urls')),

I go to https://msapiro.net/accounts/signup/ and enter email MarkSapiro@gmail.com, username MarkS and a password and SignUp.

A confirmation email is sent to MarkSapiro@gmail.com. I go to the link in the email and confirm it.

I then go to https://msapiro.net/accounts/login/ and log in as marksapiro@gmail.com with my password. At this point I am logged in as user MarkS. I then log out and log in as an admin and go to the Django admin UI at https://msapiro.net/django/account/emailaddress/ and see Screenshot from 2022-12-10 16-52-21

Note that user MarkS has two email addresses, MarkSapiro@gmail.com and marksapiro@gmail.com

At this point I can successfully log in as user MarkS with a password, but attempts to log in with either email address and the password throw this exception.

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/decorators/debug.py", line 89, in sensitive_post_parameters_wrapper
    return view(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 149, in dispatch
    return super(LoginView, self).dispatch(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 77, in dispatch
    response = super(RedirectAuthenticatedUserMixin, self).dispatch(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 105, in post
    response = self.form_valid(form)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 162, in form_valid
    return form.login(self.request, redirect_url=success_url)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/forms.py", line 196, in login
    ret = perform_login(
  File  "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 168, in perform_login
    response = adapter.pre_login(request, user, **hook_kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/adapter.py", line 411, in pre_login
    if not has_verified_email(user, email):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 130, in has_verified_email
    emailaddress = EmailAddress.objects.get_for_user(user, email)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/managers.py", line 54, in get_for_user
    ret = self.get(user=user, email__iexact=email)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py", line 439, in get
    raise self.model.MultipleObjectsReturned(

Exception Type: MultipleObjectsReturned at /accounts/login/
Exception Value: get() returned more than one EmailAddress -- it returned 2!
pennersr commented 11 months ago

The scenario you describe does not reproduce the issue for me. At what moment in your scenario does the lower case marksapiro@gmail.com email address appear in the database? It should not appear when you just login with the lower cased email address as logging in does not add email addresses, nor should it appear when you signup. Do you have any customizations in your adapter? Can you try and reproduce it with just the example project, or even better, in a test case?

pennersr commented 11 months ago

Closing as not reproducible -- feel free to reopen if you can reproduce it with an uncustomized allauth.

msapiro commented 9 months ago

I still see this issue. This time with:

Django==4.2.5
django-allauth==0.57.0

Both of these are unmodified.

The scenario is the same as previously reported, but to reiterate: 1) Go to https://msapiro.net/accounts/signup/ and sign up with Email address: Junk@msapiro.net (uppercase J) Username: junk and a password. At this point the Django admin UI Email addresses shows Screenshot from 2023-09-30 20-16-37 2) Receive the email confirmation request and follow the link therein. At this point the Django admin UI Email addresses shows Screenshot from 2023-09-30 20-17-34 3) Login at https://msapiro.net/accounts/login/ with email junk@msapiro.net and the password. Login succeeds and the Django admin UI Email addresses shows Screenshot from 2023-09-30 20-18-46 Note there are two email addresses for user junk, junk@msapiro.net and Junk@msapiro.net 4) Log out and attempt login again with email junk@msapiro.net. This fails with the traceback

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/decorators/debug.py", line 92, in sensitive_post_parameters_wrapper
    return view(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/decorators/cache.py", line 62, in _wrapper_view_func
    response = view_func(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 156, in dispatch
    return super(LoginView, self).dispatch(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 83, in dispatch
    response = super(RedirectAuthenticatedUserMixin, self).dispatch(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 111, in post
    response = self.form_valid(form)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/views.py", line 169, in form_valid
    return form.login(self.request, redirect_url=success_url)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/forms.py", line 196, in login
    ret = perform_login(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 168, in perform_login
    return _perform_login(request, login)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 178, in _perform_login
    response = adapter.pre_login(request, login.user, **hook_kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/adapter.py", line 413, in pre_login
    if not has_verified_email(user, email):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/utils.py", line 135, in has_verified_email
    emailaddress = EmailAddress.objects.get_for_user(user, email)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/managers.py", line 91, in get_for_user
    ret = self.get(user=user, email__iexact=email)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py", line 640, in get
    raise self.model.MultipleObjectsReturned(

Exception Type: MultipleObjectsReturned at /accounts/login/
Exception Value: get() returned more than one EmailAddress -- it returned 2!
Raised during: allauth.account.views.LoginView

These are the Django settings

Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Admin', 'mark@msapiro.net'),)"
ALLOWED_HOSTS = ['localhost', '127.0.0.1', 'msapiro.net', 'msg1.msapiro.net']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/opt/mailman/mm'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache', 'LOCATION': '127.0.0.1:11211'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'}
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_FILTERS = {'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'], 'js': ['compressor.filters.jsmin.rJSMinFilter']}
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7efee1d691f0>
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = True
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_MANIFEST_STORAGE = 'compressor.storage.OfflineManifestFileStorage'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/x-scss', 'sass --style compressed {infile} {outfile}'), ('text/x-sass', 'sass --style compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/opt/mailman/mm/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_URL = '/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_MASKED = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'mailman', 'USER': 'mark', 'PASSWORD': '********************', 'HOST': '127.0.0.1', 'PORT': '', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'CONN_HEALTH_CHECKS': False, 'OPTIONS': {}, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'MIGRATE': True, 'MIRROR': None, 'NAME': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATA_UPLOAD_MAX_NUMBER_FILES = 100
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_EXCEPTION_REPORTER = 'django.views.debug.ExceptionReporter'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'mark@msapiro.net'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_CONFIRMATION_FROM = 'mark@msapiro.net'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = 420
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'xapian_backend.XapianEngine', 'PATH': '/opt/mailman/mm/var/xapian_index'}}
HYPERKITTY_ALLOW_WEB_POSTING = True
HYPERKITTY_DISABLE_SINGLETON_TASKS = True
HYPERKITTY_ENABLE_GRAVATAR = True
HYPERKITTY_JOBS_UPDATE_INDEX_LOCK_LIFE = 900
HYPERKITTY_TASK_LOCK_TIMEOUT = 600
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'allauth.socialaccount.providers.github', 'allauth.socialaccount.providers.gitlab', 'allauth.socialaccount.providers.google', 'allauth.socialaccount.providers.facebook')"
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ar-dz', 'Algerian Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('ckb', 'Central Kurdish (Sorani)'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('hy', 'Armenian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('ig', 'Igbo'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kab', 'Kabyle'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('ky', 'Kyrgyz'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('ms', 'Malay'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('tg', 'Tajik'), ('th', 'Thai'), ('tk', 'Turkmen'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('uz', 'Uzbek'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'ar-dz', 'ckb', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_HTTPONLY = False
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LANGUAGE_COOKIE_SAMESITE = None
LANGUAGE_COOKIE_SECURE = False
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.WatchedFileHandler', 'filename': '/opt/mailman/mm/logs/mailmansuite.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'ERROR', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'ERROR', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'DEBUG', 'propagate': True}, 'postorius': {'handlers': ['console', 'file'], 'level': 'INFO'}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1', '::ffff:127.0.0.1', '45.24.217.241')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = '/'
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('allauth.account.middleware.AccountMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')"
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT = '********************'
POSTORIUS_TEMPLATE_BASE_URL = 'https://msapiro.net'
PREPEND_WWW = False
Q_CLUSTER = {'timeout': 300, 'retry': 360, 'save_limit': 100, 'orm': 'default'}
ROOT_URLCONF = 'urls'
SECRET_KEY = '********************'
SECRET_KEY_FALLBACKS = '********************'
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_CROSS_ORIGIN_OPENER_POLICY = 'same-origin'
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = "('HTTP_X_FORWARDED_PROTO', 'https')"
SECURE_REDIRECT_EXEMPT = []
SECURE_REFERRER_POLICY = 'same-origin'
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'mark@msapiro.net'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SAMESITE = 'Lax'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SHOW_ANONYMOUS_SUBSCRIBE_FORM = True
SHOW_INACTIVE_LISTS_DEFAULT = False
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 0
SOCIALACCOUNT_PROVIDERS = {'google': {'SCOPE': ['profile', 'email'], 'AUTH_PARAMS': {'access_type': 'online'}}, 'facebook': {'METHOD': 'oauth2', 'SCOPE': ['email'], 'FIELDS': ['email', 'name', 'first_name', 'last_name', 'locale', 'timezone'], 'VERSION': 'v2.12'}, 'gitlab': {'SCOPE': ['read_user']}}
STATICFILES_DIRS = '()'
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/opt/mailman/mm/static'
STATIC_URL = '/static/'
STORAGES = {'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'}, 'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage'}}
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_DEPRECATED_PYTZ = False
USE_I18N = True
USE_L10N = True
USE_SSL = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = True
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'wsgi.application'
X_FRAME_OPTIONS = 'DENY'
YEAR_MONTH_FORMAT = 'F Y'
msapiro commented 9 months ago

I can't reopen this as I am not a collaborator in this project so I opened #3459 pointing to this.

pennersr commented 9 months ago

When using the example project with your configuration:

ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True

... I still cannot reproduce this. Your settings show that you are running many more apps than just allauth, which makes it unclear that this is an issue in allauth itself.

So, these remarks still stand:

Can you try and reproduce it with just the example project, or even better, in a test case?

Closing as not reproducible -- feel free to reopen if you can reproduce it with an uncustomized allauth.

pennersr commented 9 months ago

It's an issue in django_mailman3. See:

https://gitlab.com/mailman/django-mailman3/-/blob/master/django_mailman3/signals.py?ref_type=heads#L111

https://gitlab.com/mailman/django-mailman3/-/blob/master/django_mailman3/lib/mailman.py?ref_type=heads#L216

It looks up email addresses in a case sensitive manner.

Now, you could work around this by adding this to your adapter:

    def clean_email(self, email):
        return email.lower()
pennersr commented 9 months ago

Keep an eye on -- or better, give it a spin: #3460

msapiro commented 9 months ago

@pennersr, Thank you very much for your analysis and for #3460. I appreciate this very much. I do understand what RFC 5321 and predecessors say about interpretation of local-part, and how these are hard choices. I will be looking at this in more detail over the next days and will report back.

msapiro commented 9 months ago

I have tested with #3460 installed and it seems fine and solves my issue. Thank you.

msapiro commented 9 months ago

I may have spoken too soon. On one server that had a couple of users with 2 email addresses differing only in case, when I installed #3460, the account.0006_emailaddress_lower migration failed with

  Applying account.0006_emailaddress_lower...Traceback (most recent call last):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/dbapi/__init__.py", line 112, in execute
    return self._trace_method(self.__wrapped__.execute, self._self_datadog_name, query, {}, query, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/psycopg/patch.py", line 58, in _trace_method
    return super(Psycopg2TracedCursor, self)._trace_method(method, name, resource, extra_tags, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/dbapi/__init__.py", line 75, in _trace_method
    return method(*args, **kwargs)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "account_emailaddress_user_id_email_987c8728_uniq"
DETAIL:  Key (user_id, email)=(43733, hgeom.app.inventor@gmail.com) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module>
    sys.exit(execute_from_command_line())
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 244, in handle
    post_migrate_state = executor.migrate(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/migrations/executor.py", line 227, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/migrations/migration.py", line 126, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/account/migrations/0006_emailaddress_lower.py", line 11, in forwards
    EmailAddress.objects.all().exclude(email=Lower("email")).update(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py", line 783, in update
    rows = query.get_compiler(self.db).execute_sql(CURSOR)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1559, in execute_sql
    cursor = super().execute_sql(result_type)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
    cursor.execute(sql, params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/dbapi/__init__.py", line 112, in execute
    return self._trace_method(self.__wrapped__.execute, self._self_datadog_name, query, {}, query, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/psycopg/patch.py", line 58, in _trace_method
    return super(Psycopg2TracedCursor, self)._trace_method(method, name, resource, extra_tags, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/dbapi/__init__.py", line 75, in _trace_method
    return method(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/raven/contrib/django/client.py", line 127, in execute
    return real_execute(self, sql, params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 545, in execute
    return real_execute(self, sql, params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/dbapi/__init__.py", line 112, in execute
    return self._trace_method(self.__wrapped__.execute, self._self_datadog_name, query, {}, query, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/psycopg/patch.py", line 58, in _trace_method
    return super(Psycopg2TracedCursor, self)._trace_method(method, name, resource, extra_tags, *args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/ddtrace/contrib/dbapi/__init__.py", line 75, in _trace_method
    return method(*args, **kwargs)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "account_emailaddress_user_id_email_987c8728_uniq"
DETAIL:  Key (user_id, email)=(43733, hgeom.app.inventor@gmail.com) already exists.

I then ran my script which finds users with emails differing only in case and deletes the mixed case one and sets the all lower case one primary, and after that the migrations ran successfully.

This may not be important as the condition with multiple emails differing only in case was due ti issues in django-mailman3.

pennersr commented 9 months ago

@msapiro Thanks for the feedback. Besides documenting in the release notes that this might happen there is not much one can do to automatically resolve such conflicts.