[5pt] Some integration-tests for aws&smaug are red or can be turned off

thoth-station / integration-tests

Integration tests for the Thoth project to make sure deployment works as expected

GNU General Public License v3.0

4 stars 12 forks source link

[5pt] Some integration-tests for aws&smaug are red or can be turned off #302

Open fridex opened 2 years ago

fridex commented 2 years ago

Describe the bug

See the most recent report from the AWS&Smaug clusters. We should be able to reproduce the same quality of report for the AWS cluster as for the stage environment.

To Reproduce

See the most recent report for the AWS cluster.

For the test cases we do not want to run in AWS&Smaug clusters, we should set up behave tags.

fridex commented 2 years ago

Failing tests

[ ] Scenario: Browsing logs of container images builds during Amun inspection jobs/Then wait for inspection to finish successfully
[ ] Dependency Monkey tests
[ ] Container image analysis
[ ] Query for metadata for different indices for TensorFlow
[ ] Querying for the known Python Package Indexes on User API - most probably requires update of the database
- missing https://pulp.operate-first.cloud/pypi/gym-donkeycar/simple
- missing https://pulp.operate-first.cloud/pypi/hello-world/simple
- missing https://pulp.operate-first.cloud/pypi/test/simple
[ ] schedule solver job
[ ] advise scenarios failing with HTTP 400
[ ] provenance scenarios failing with HTTP 400

fridex commented 2 years ago

/priority critical-urgent

Gregory-Pereira commented 2 years ago

/assign

mayaCostantini commented 2 years ago

/assign

codificat commented 2 years ago

/triage accepted

fridex commented 2 years ago

Based on the discussion we had with @harshad16, we will point integration tests to prod instead of smaug so results will differ. Let's wait for the report sent.

fridex commented 2 years ago

Related action item to be done first: https://github.com/thoth-station/thoth-application/issues/2451

Gregory-Pereira commented 2 years ago

once the documentation pr gets merged, this will be no longer be blocked by thoth-station/thoth-application#2451.

codificat commented 2 years ago

/sig devsecops

codificat commented 2 years ago

actually this is /sig stack-guidance to confirm the status of the various failing scenarios

codificat commented 2 years ago

/remove-sig devsecops

sesheta commented 2 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

harshad16 commented 2 years ago

/remove-lifecycle stale /lifecycle frozen

KPostOffice commented 1 year ago

Running tests and looking into failing ones in stage. Running tests takes multiple hours, so this is slow going.