opendatahub-io / model-registry-bf4-kf

Apache License 2.0
1 stars 10 forks source link

[model-controller] Setup e2e tests for model registry and serving reconciliation #235

Open lampajr opened 10 months ago

lampajr commented 10 months ago

Is your feature request related to a problem? Please describe. In the first implementation of model registry and serving implemention we focused on creating unit testing by mocking the model registry service because the TestContainer approach was not feasible as it would not work in openshift-ci environment [1]. On the other hand having some ITs where we can actually test the connection to an existing model registry/ml-metadata service could be very helpful and furthermore we could setup real use cases in these tests.

[1] https://cloud.redhat.com/blog/running-testcontainers-in-openshift-pipelines-with-docker-in-docker

Describe the solution you'd like Setup e2e test suite in odh-model-controller where we could setup model-registry service by directly applying a deployment and then run all expected tests.

Describe alternatives you've considered Keep just unit tests, unfortunately we these we cannot cover all possible tests (especially those that writes on model registry are not really testable as we are mocking it)

Additional context Some examples we could take inspiration from:

lampajr commented 10 months ago

Investigating how to implement those e2e tests in the mode generic fashion, such that they will work regardless the infrastructure will actually run them (either openshift_ci or gha).

Tests assumption: