Closed goern closed 3 years ago
Describe the bug
Feature: Check for minimum solvers available in Thoth # features/solvers.feature:1 Scenario: Check there are minimum solvers available in Thoth # features/solvers.feature:2 Given deployment is accessible using HTTPS # features/steps/basic.py:31 1.106s When we ask for the available solvers # features/steps/solvers.py:30 0.591s Then they should include at least the minimum set of solvers # features/steps/solvers.py:39 0.001s Scenario: Schedule solver jobs for all available solvers in Thoth # features/solvers.feature:7 Given deployment is accessible using HTTPS # features/steps/basic.py:31 1.095s When we ask for the available solvers # features/steps/solvers.py:30 0.556s Then schedule solver analyses for package selinon with version 1.0.0 # features/steps/solvers.py:47 0.848s Then wait for analyses to finish successfully # features/steps/solvers.py:64 135.990s Assertion Failed: Solver solver-rhel-8-py36-d20ef8e118cae6b9d76e4db031864ec9 run on management.stage.thoth-station.ninja was not successful
To Reproduce Steps to reproduce the behavior:
- run integration-tests
/kind bug /label sig/devops
I rescheduled same package
Name: solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce
Namespace: thoth-middletier-stage
ServiceAccount: argo
Status: Failed
Conditions:
Completed True
Created: Thu Jan 14 12:56:14 +0100 (3 minutes ago)
Started: Thu Jan 14 12:56:14 +0100 (3 minutes ago)
Finished: Thu Jan 14 12:59:39 +0100 (now)
Duration: 3 minutes 25 seconds
ResourcesDuration: 3m23s*(1 cpu),12m49s*(100Mi memory)
Parameters:
ceph_bucket_prefix: data
ceph_bucket_name: thoth
ceph_host: s3.upshift.redhat.com
deployment_name: ocp4-stage
THOTH_SOLVER_NAME: solver-rhel-8-py36
THOTH_SOLVER_WORKFLOW_ID: solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce
THOTH_LOG_SOLVER: INFO
THOTH_SOLVER_NO_TRANSITIVE: 1
THOTH_SOLVER_PACKAGES: selinon==1.0.0
THOTH_SOLVER_INDEXES: https://pypi.org/simple
THOTH_FORCE_SYNC: 0
STEP TEMPLATE PODNAME DURATION MESSAGE
✖ solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce solve-and-sync
├-✔ solverany(0) solver-any/solve solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-112752507 30s
├-✖ graph-sync-solverany graph-sync/graph-sync No more retries left
| ├-✖ graph-sync-solverany(0) graph-sync/graph-sync solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-4184177582 54s failed with exit code 1
| ├-✖ graph-sync-solverany(1) graph-sync/graph-sync solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-3580036203 33s failed with exit code 1
| ├-✖ graph-sync-solverany(2) graph-sync/graph-sync solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1969929232 33s failed with exit code 1
| └-✖ graph-sync-solverany(3) graph-sync/graph-sync solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757 43s failed with exit code 1
The issue is related to graph-sync-job, solver task succeeded, but the graph-sync-job cannot be performed by design for package that we already solved to prevent from duplicates.
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.780162303Z Traceback (most recent call last):
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.783238253Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z cursor.execute(statement, parameters)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "python_package_version_package_name_package_version_python__key"
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z DETAIL: Key (package_name, package_version, python_package_index_id, os_name, os_version, python_version)=(selinon, 1.0.0, 1, rhel, 8, 3.6) already exists.
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z The above exception was the direct cause of the following exception:
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z File "/opt/app-root/lib64/python3.8/site-packages/thoth/storages/graph/models_base.py", line 52, in get_or_create
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z Traceback (most recent call last):
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z self.dialect.do_execute(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z session.commit()
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 1042, in commit
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z self.transaction.commit()
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 504, in commit
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z self._prepare_impl()
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.78593985Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.866682108Z self.session.flush()
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.866682108Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 2536, in flush
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.867475608Z self._flush(objects)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.867475608Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 2678, in _flush
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.868434293Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.868434293Z transaction.rollback(_capture_exception=True)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.868565506Z compat.raise_(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.868565506Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.868689383Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 2638, in _flush
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.868689383Z raise exception
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869301912Z flush_context.execute()
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869301912Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869594987Z rec.execute(self)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869594987Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869837203Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869837203Z persistence.save_obj(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869962047Z _emit_insert_statements(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.869962047Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.870270065Z result = cached_connections[connection].execute(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.870270065Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.870564513Z return meth(self, multiparams, params)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.870564513Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.870722939Z return connection._execute_clauseelement(self, multiparams, params)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.870722939Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.871019209Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.871019209Z ret = self._execute_context(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z self._handle_dbapi_exception(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z util.raise_(
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z raise exception
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z return session.query(cls).filter_by(**kwargs).one(), True
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z cursor.execute(statement, parameters)
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "python_package_version_package_name_package_version_python__key"
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z DETAIL: Key (package_name, package_version, python_package_index_id, os_name, os_version, python_version)=(selinon, 1.0.0, 1, rhel, 8, 3.6) already exists.
solver-rhel-8-py36-c219b7045b32333b9efc8a8fe2e4b9ce-1634229757: 2021-01-14T11:59:29.882580835Z
We need to modify the integration test accordingly or the endpoint we use. @goern @fridex
if it is run on newly created database from init-job, this test would not fail.
I wonder if this behavior is expected though. If we would want to rerun the solver with the same configuration for some reason (e.g. fix in solver), the most recent solver results should be taken into account in adviser, thus this syncing should proceed even if the given package was already seen in the previous solver runs.
I wonder if this behavior is expected though. If we would want to rerun the solver with the same configuration for some reason (e.g. fix in solver), the most recent solver results should be taken into account in adviser, thus this syncing should proceed even if the given package was already seen in the previous solver runs.
In that case, shouldn't we just force the sync?
In that case, shouldn't we just force the sync?
The duplicate key violation on unique constraint set up in the database will raise this exception regardless of force sync used in our code.
In that case, shouldn't we just force the sync?
The duplicate key violation on unique constraint set up in the database will raise this exception regardless of force sync used in our code.
yeah right sorry :)
/assign @pacospace
The scenario succeeds, all the issues stated are fixed now. Closing, feel free to reopen if you spot any other issue.
Describe the bug
To Reproduce Steps to reproduce the behavior:
/kind bug /label sig/devops