pulp / pulpcore

Pulp 3 pulpcore package https://pypi.org/project/pulpcore/
GNU General Public License v2.0
292 stars 113 forks source link

[Epic] Improve functional tests accross pulpcore and plugins #1890

Open pulpbot opened 2 years ago

pulpbot commented 2 years ago

Author: @dkliban (dkliban@redhat.com)

Redmine Issue: 6637, https://pulp.plan.io/issues/6637


Motivation

Pulp is designed to work with a lot of different content that's comprised of different artifacts. While some functional tests are specifically designed to check how Pulp handles content that it already has in its storage, the vast majority of tests assume that the content being used by the test is unique. Due to the limited number of test fixtures, many of the tests run the orphan clenup task to ensure that any content left over from a previous test is removed before the current test is executed. Not only does this add a significant amount of time to the overall runtime of the test suite, but it also creates indeterministic results when the tests are run in a random order.

Solution

In order to make each test independent of other tests, each test should come with its own set of unique fixtures. These fixtures should live in the same repository as the tests. The pulplift dev environment should be updated to launch a fixture container that volume mounts the fixtures from all the pulp repositories. These volume mounts should then be made available over http using nginx.

stale[bot] commented 2 years ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!