search

cgwire / zou

Zou is the Kitsu API. It allows you to store and manage your production data
https://zou.cg-wire.com
GNU Affero General Public License v3.0
166 stars 103 forks source link

Partial Project deletion bug, now cannot delete it anymore! #808

Open Ramel opened 3 months ago

Ramel commented 3 months ago

Context

Studio name: Tchack Zou version: 0.19.12 Zou installation type: self-hosted

Describe the bug

I've deleted a project in the web interface, but the deletion does not reach its end, I suppose, as the project is still there after deletion. The server as hang during the deletion (disk full and memory also) and I've rebooted it. And now I'm left with an empty project, but I cannot delete it.

In Gazu if I do:

remove_project = gazu.project.remove_project('0b786dd5-9870-4518-8213-6a9db0cb6178')
print("remove_project = {}".format(remove_project))

It returns:

A server error occured!

Server stacktrace:
Traceback (most recent call last):
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1970, in _exec_single_context
    self.dialect.do_execute(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
  File "/opt/zouenv/lib/python3.10/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.ForeignKeyViolation: update or delete on table "project" violates foreign key constraint "metadata_descriptor_project_id_fkey" on table "metadata_descriptor"
DETAIL:  Key (id)=(0b786dd5-9870-4518-8213-6a9db0cb6178) is still referenced from table "metadata_descriptor".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/zouenv/lib/python3.10/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/zouenv/lib/python3.10/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/opt/zouenv/lib/python3.10/site-packages/flask_restful/__init__.py", line 489, in wrapper
    resp = resource(*args, **kwargs)
  File "/opt/zouenv/lib/python3.10/site-packages/flask/views.py", line 110, in view
    return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
  File "/opt/zouenv/lib/python3.10/site-packages/flask_restful/__init__.py", line 604, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/opt/zouenv/lib/python3.10/site-packages/flask_jwt_extended/view_decorators.py", line 170, in decorator
    return current_app.ensure_sync(fn)(*args, **kwargs)
  File "/opt/zouenv/lib/python3.10/site-packages/zou/app/blueprints/crud/project.py", line 189, in delete
    project.delete()
  File "/opt/zouenv/lib/python3.10/site-packages/zou/app/models/base.py", line 210, in delete
    db.session.commit()
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/scoping.py", line 597, in commit
    return self._proxied.commit()
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1972, in commit
    trans.commit(_to_root=True)
  File "<string>", line 2, in commit
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1257, in commit
    self._prepare_impl()
  File "<string>", line 2, in _prepare_impl
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1232, in _prepare_impl
    self.session.flush()
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 4296, in flush
    self._flush(objects)
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 4431, in _flush
    with util.safe_reraise():
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 4392, in _flush
    flush_context.execute()
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 679, in execute
    util.preloaded.orm_persistence.delete_obj(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 193, in delete_obj
    _emit_delete_statements(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1465, in _emit_delete_statements
    c = connection.execute(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1421, in execute
    return meth(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 514, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1643, in _execute_clauseelement
    ret = self._execute_context(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1849, in _execute_context
    return self._exec_single_context(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1989, in _exec_single_context
    self._handle_dbapi_exception(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2356, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1970, in _exec_single_context
    self.dialect.do_execute(
  File "/opt/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
  File "/opt/zouenv/lib/python3.10/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
sqlalchemy.exc.IntegrityError: (psycopg.errors.ForeignKeyViolation) update or delete on table "project" violates foreign key constraint "metadata_descriptor_project_id_fkey" on table "metadata_descriptor"
DETAIL:  Key (id)=(0b786dd5-9870-4518-8213-6a9db0cb6178) is still referenced from table "metadata_descriptor".
[SQL: DELETE FROM project WHERE project.id = %(id)s::UUID]
[parameters: {'id': '0b786dd5-9870-4518-8213-6a9db0cb6178'}]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

Error message:
(psycopg.errors.ForeignKeyViolation) update or delete on table "project" violates foreign key constraint "metadata_descriptor_project_id_fkey" on table "metadata_descriptor"
DETAIL:  Key (id)=(0b786dd5-9870-4518-8213-6a9db0cb6178) is still referenced from table "metadata_descriptor".
[SQL: DELETE FROM project WHERE project.id = %(id)s::UUID]
[parameters: {'id': '0b786dd5-9870-4518-8213-6a9db0cb6178'}]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

Traceback (most recent call last):
  File "c:\Users\Dev\Gazu\list_projects.py", line 25, in <module>
    remove_project = gazu.project.remove_project('0b786dd5-9870-4518-8213-6a9db0cb6178')
  File "C:\Users\Dev\.pyenv\pyenv-win\versions\3.9.13\lib\site-packages\gazu\project.py", line 154, in remove_project
    return raw.delete(path, client=client)
  File "C:\Users\Dev\.pyenv\pyenv-win\versions\3.9.13\lib\site-packages\gazu\client.py", line 272, in delete
    check_status(response, path)
  File "C:\Users\Dev\.pyenv\pyenv-win\versions\3.9.13\lib\site-packages\gazu\client.py", line 326, in check_status
    raise ServerErrorException(path)
gazu.exception.ServerErrorException: data/projects/0b786dd5-9870-4518-8213-6a9db0cb6178

Expected behavior Find a way if possible to delete the project without breaking the BDD. I don't know if the previews are also deleted for the project. I can still leave it as "Closed" but I prefer to delete the project to room some space on the server.

frankrousseau commented 2 weeks ago

Can you set the force flag to True ?

Ramel commented 2 weeks ago

With force flag set to True :

A server error occured!

Server stacktrace:
Traceback (most recent call last):
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1970, in _exec_single_context
    self.dialect.do_execute(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
  File "/home/zou/zouenv/lib/python3.10/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.ForeignKeyViolation: update or delete on table "metadata_descriptor" violates foreign key constraint "department_metadata_descriptor_link_metadata_descriptor_id_fkey" on table "department_metadata_descriptor_link"
DETAIL:  Key (id)=(7f9111d4-8d4b-4484-a8e8-1c58cb0f669d) is still referenced from table "department_metadata_descriptor_link".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/zou/zouenv/lib/python3.10/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/zou/zouenv/lib/python3.10/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/home/zou/zouenv/lib/python3.10/site-packages/flask_restful/__init__.py", line 489, in wrapper
    resp = resource(*args, **kwargs)
  File "/home/zou/zouenv/lib/python3.10/site-packages/flask/views.py", line 110, in view
    return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
  File "/home/zou/zouenv/lib/python3.10/site-packages/flask_restful/__init__.py", line 604, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/home/zou/zouenv/lib/python3.10/site-packages/flask_jwt_extended/view_decorators.py", line 170, in decorator
    return current_app.ensure_sync(fn)(*args, **kwargs)
  File "/home/zou/zouenv/lib/python3.10/site-packages/zou/app/blueprints/crud/project.py", line 187, in delete
    deletion_service.remove_project(instance_id)
  File "/home/zou/zouenv/lib/python3.10/site-packages/zou/app/services/deletion_service.py", line 363, in remove_project
    MetadataDescriptor.delete_all_by(project_id=project_id)
  File "/home/zou/zouenv/lib/python3.10/site-packages/zou/app/models/base.py", line 133, in delete_all_by
    result = cls.query.filter_by(**kw).delete()
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3161, in delete
    result: CursorResult[Any] = self.session.execute(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2306, in execute
    return self._execute_internal(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2191, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/bulk_persistence.py", line 1946, in orm_execute_statement
    return super().orm_execute_statement(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement
    result = conn.execute(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1421, in execute
    return meth(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 514, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1643, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1849, in _execute_context
    return self._exec_single_context(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1989, in _exec_single_context
    self._handle_dbapi_exception(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2356, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1970, in _exec_single_context
    self.dialect.do_execute(
  File "/home/zou/zouenv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
  File "/home/zou/zouenv/lib/python3.10/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
sqlalchemy.exc.IntegrityError: (psycopg.errors.ForeignKeyViolation) update or delete on table "metadata_descriptor" violates foreign key constraint "department_metadata_descriptor_link_metadata_descriptor_id_fkey" on table "department_metadata_descriptor_link"
DETAIL:  Key (id)=(7f9111d4-8d4b-4484-a8e8-1c58cb0f669d) is still referenced from table "department_metadata_descriptor_link".
[SQL: DELETE FROM metadata_descriptor WHERE metadata_descriptor.project_id = %(project_id_1)s::UUID]
[parameters: {'project_id_1': '0b786dd5-9870-4518-8213-6a9db0cb6178'}]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

Error message:
(psycopg.errors.ForeignKeyViolation) update or delete on table "metadata_descriptor" violates foreign key constraint "department_metadata_descriptor_link_metadata_descriptor_id_fkey" on table "department_metadata_descriptor_link"
DETAIL:  Key (id)=(7f9111d4-8d4b-4484-a8e8-1c58cb0f669d) is still referenced from table "department_metadata_descriptor_link".
[SQL: DELETE FROM metadata_descriptor WHERE metadata_descriptor.project_id = %(project_id_1)s::UUID]
[parameters: {'project_id_1': '0b786dd5-9870-4518-8213-6a9db0cb6178'}]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

Traceback (most recent call last):
  File "c:\Users\Dev\Gazu\list_projects.py", line 30, in <module>
    remove_project = gazu.project.remove_project('0b786dd5-9870-4518-8213-6a9db0cb6178',force=True)
  File "c:\Users\Dev\.pyenv\pyenv-win\versions\3.9.13\lib\site-packages\gazu\project.py", line 154, in remove_project
    return raw.delete(path, client=client)
  File "c:\Users\Dev\.pyenv\pyenv-win\versions\3.9.13\lib\site-packages\gazu\client.py", line 272, in delete
    check_status(response, path)
  File "c:\Users\Dev\.pyenv\pyenv-win\versions\3.9.13\lib\site-packages\gazu\client.py", line 326, in check_status
    raise ServerErrorException(path)
gazu.exception.ServerErrorException: data/projects/0b786dd5-9870-4518-8213-6a9db0cb6178?force=true
frankrousseau commented 3 days ago

Ok it will be fixed in the next release.