fedora-infra / fmn

A system for generic fedmsg-driven notifications for end users.
25 stars 32 forks source link

FMN fails to edit rule #922

Closed gotmax23 closed 1 year ago

gotmax23 commented 1 year ago

I'm trying to edit a rule. FMN is sending a PUT request to https://fmn-api.apps.ocp.fedoraproject.org/api/v1/users/gotmax23/rules/XX and then failing. I've attached a screenshot of the error console:

FMN Edit Rule Failure Console

abompard commented 1 year ago

This looks like the backend tracebacking and as a result not sending the CORS headers. I'll look a the logs, thanks for the report.

abompard commented 1 year ago

Interesting, we're getting:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  [...SNIP...]
asyncpg.exceptions.SerializationError: could not serialize access due to concurrent update

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...SNIP...]
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.Error: <class 'asyncpg.exceptions.SerializationError'>: could not serialize access due to concurrent update

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...SNIP...]
  File "/opt/app-root/src/fmn/api/handlers/users.py", line 186, in edit_user_rule
    await db_session.flush()
  [...SNIP...]
sqlalchemy.exc.DBAPIError: (sqlalchemy.dialects.postgresql.asyncpg.Error) <class 'asyncpg.exceptions.SerializationError'>: could not serialize access due to concurrent update
[SQL: UPDATE rules SET name=$1::VARCHAR WHERE rules.id = $2::INTEGER]
[parameters: ('Finished Koji and Copr build (Mine)', 57)]
(Background on this error at: https://sqlalche.me/e/20/dbapi)
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  [...SNIP...]
asyncpg.exceptions.SerializationError: could not serialize access due to read/write dependencies among transactions
DETAIL:  Reason code: Canceled on identification as a pivot, during write.
HINT:  The transaction might succeed if retried.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...SNIP...]
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.Error: <class 'asyncpg.exceptions.SerializationError'>: could not serialize access due to read/write dependencies among transactions
DETAIL:  Reason code: Canceled on identification as a pivot, during write.
HINT:  The transaction might succeed if retried.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...SNIP...]
  File "/opt/app-root/src/fmn/api/handlers/users.py", line 186, in edit_user_rule
    await db_session.flush()
  [...SNIP...]
sqlalchemy.exc.DBAPIError: (sqlalchemy.dialects.postgresql.asyncpg.Error) <class 'asyncpg.exceptions.SerializationError'>: could not serialize access due to read/write dependencies among transactions
DETAIL:  Reason code: Canceled on identification as a pivot, during write.
HINT:  The transaction might succeed if retried.
[SQL: UPDATE filters SET params=$1::JSON WHERE filters.id = $2::INTEGER]
[parameters: ('["Koji", "Copr"]', 236)]
(Background on this error at: https://sqlalche.me/e/20/dbapi)

(I snipped calls inside frameworks and libraries)

@nphilipp , do you want to have a look? Otherwise I can, but you know async better than I do :-)

nphilipp commented 1 year ago

Hm, looks like a conflict between DB transactions to me, perhaps a user was attempting to edita rule while the rule cache was being refreshed? If so, maybe we should lower the transaction isolation level for the latter as it takes some time and consistency between such transactions isn't super important.