Crash when accepting/rejecting multiple applications

[mome-borogove]

Trigger: when on the applications tab, clicking more than one action causes a crash (presumably---it could technically be something else, I suppose)

IntegrityError at /whctools/staff
(1062, "Duplicate entry '486-1' for key 'memberaudit_characterskillsetcheck.functional_pk_characterskillsetcheck'")
Request Method: GET
Request URL:    http://auth.eveuniversity.org/whctools/staff
Django Version: 4.0.10
Exception Type: IntegrityError
Exception Value:    
(1062, "Duplicate entry '486-1' for key 'memberaudit_characterskillsetcheck.functional_pk_characterskillsetcheck'")
Exception Location: /opt/venv/lib/python3.9/site-packages/MySQLdb/connections.py, line 255, in query
Python Executable:  /opt/venv/bin/python3
Python Version: 3.9.18
Python Path:    
['/home/allianceauth/myauth',
 '/home/allianceauth/myauth',
 '/opt/venv/bin',
 '/usr/local/lib/python39.zip',
 '/usr/local/lib/python3.9',
 '/usr/local/lib/python3.9/lib-dynload',
 '/opt/venv/lib/python3.9/site-packages']
Server time:    Sat, 22 Jun 2024 21:01:03 +0000

Environment:

Request Method: GET
Request URL: http://auth.eveuniversity.org/whctools/staff

Django Version: 4.0.10
Python Version: 3.9.18
Installed Applications:
['allianceauth',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django_celery_beat',
 'bootstrapform',
 'sortedm2m',
 'esi',
 'allianceauth.authentication',
 'allianceauth.services',
 'allianceauth.eveonline',
 'allianceauth.groupmanagement',
 'allianceauth.notifications',
 'allianceauth.thirdparty.navhelper',
 'allianceauth.analytics',
 'eunicore',
 'allianceauth.services.modules.discord',
 'allianceauth.services.modules.mumble',
 'allianceauth.services.modules.phpbb3',
 'allianceauth.permissions_tool',
 'aasrp',
 'allianceauth.corputils',
 'allianceauth_pve',
 'blacklist',
 'buybackprogram',
 'discordnotify',
 'celeryanalytics',
 'eveuniverse',
 'fleetpings',
 'freight',
 'groupassign',
 'incursions',
 'marketmanager',
 'memberaudit',
 'memberaudit_securegroups',
 'moonstuff',
 'securegroups',
 'structures',
 'structuretimers',
 'taskmonitor',
 'package_monitor',
 'afat',
 'opcalendar',
 'fittings',
 'membertools',
 'whctools']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'allianceauth.authentication.middleware.UserSettingsMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'allianceauth.analytics.middleware.AnalyticsMiddleware']

Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
  File "/opt/venv/lib/python3.9/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
  File "/opt/venv/lib/python3.9/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/opt/venv/lib/python3.9/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)

The above exception ((1062, "Duplicate entry '486-1' for key 'memberaudit_characterskillsetcheck.functional_pk_characterskillsetcheck'")) was the direct cause of the following exception:
  File "/opt/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/opt/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/venv/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/allianceauth/authentication/decorators.py", line 47, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/whctools/views.py", line 171, in staff
    ma_character.update_skill_sets()
  File "/opt/venv/lib/python3.9/site-packages/memberaudit/models/characters.py", line 549, in update_skill_sets
    self.skill_set_checks.update_for_character(self)
  File "/usr/local/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/opt/venv/lib/python3.9/site-packages/memberaudit/managers/character_sections_3.py", line 404, in update_for_character
    self.bulk_create(
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py", line 579, in bulk_create
    returned_columns = self._batched_insert(
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1467, in _batched_insert
    self._insert(
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1434, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1621, in execute_sql
    cursor.execute(sql, params)
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/venv/lib/python3.9/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/venv/lib/python3.9/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
  File "/opt/venv/lib/python3.9/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
  File "/opt/venv/lib/python3.9/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/opt/venv/lib/python3.9/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)

Exception Type: IntegrityError at /whctools/staff
Exception Value: (1062, "Duplicate entry '486-1' for key 'memberaudit_characterskillsetcheck.functional_pk_characterskillsetcheck'")

[lynkfox]

how do you click more than one action?

from this page?

all those buttons should reload the page directly and you shouldnt be able to click more than one action?

[lynkfox]

and on the rejection page, i am not able to duplicate this bug. Can you provide more information?

(I am suspecting however theyre may have been a database corruption issue here with somehow it getting more than one of the same entry)

[mome-borogove]

how do you click more than one action? all those buttons should reload the page directly and you shouldnt be able to click more than one action?

"reload the page" is a 20-30 second operation. During this time, every part of the interface is active and sends commands which are received by the backend as normal. For the applications tab, if there's six characters, you can easily click a dozen or more buttons. This is actually really useful on the cooldown reset page, since you can reset every character at once. Otherwise, resetting my alts takes 6-7 minutes. It's painful.

Can you provide more information? I suspect it's due to issuing multiple commands that have conflicting outcomes. You're testing with a single character. Try a couple alts, then accept and reject several at a time.

If the answer is "wait for the page to reload", I guess that's fine. But the load time would need to be at least 10x faster.

[lynkfox]

Good to know that makes more sense.

I've already started work on breaking up the page (right now all the staff pages are technically one page) and that will help.

I've got some other plans for optimization

On Mon, Jun 24, 2024, 1:52 PM mome-borogove @.***> wrote:

how do you click more than one action? all those buttons should reload the page directly and you shouldnt be able to click more than one action?

"reload the page" is a 20-30 second operation. During this time, every part of the interface is active and sends commands which are received by the backend as normal. For the applications tab, if there's six characters, you can easily click a dozen or more buttons. This is actually really useful on the cooldown reset page, since you can reset every character at once. Otherwise, resetting my alts takes 6-7 minutes. It's painful.

Can you provide more information? I suspect it's due to issuing multiple commands that have conflicting outcomes. You're testing with a single character. Try a couple alts, then accept and reject several at a time.

If the answer is "wait for the page to reload", I guess that's fine. But the load time would need to be at least 10x faster.

— Reply to this email directly, view it on GitHub https://github.com/EVE-University/whctools/issues/10#issuecomment-2187105017, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADLOBQ6SXYKXX7R7MXYWRJ3ZJBMFJAVCNFSM6AAAAABJXZ5J72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGEYDKMBRG4 . You are receiving this because you commented.Message ID: @.***>

[lynkfox]

should have this (and the 20-30 second load time) fixed in the current version im working on.

[mome-borogove]

Can't repro this, and it's a hell of a lot harder to set up the conditions which it required, too. Considering it closed.