[EPIC] Add regression/functional tests for booster projects

fabric8io / fabric8-maven-plugin

📢 This project is migrated to 👉 https://github.com/eclipse/jkube

Apache License 2.0

334 stars 201 forks source link

[EPIC] Add regression/functional tests for booster projects #1103

Closed hrishin closed 6 years ago

hrishin commented 6 years ago

At present, there is no regression test exist to verify the deployment of booster/starter applications [1] [2] on Kubernetes and OpenShift environment. Any release or PR can go through a regression test for plugin project that can make sure application deployment is functioning well.

Applications to deploy: [1] VertX [2] Springboot

The broader idea is to have regression test suit that can run on PR submission of plugin, kubernetes-client and kubernetes-model projects to test the deployment of booster applications[1][2]

Tasks:

[x] Identify the testing framework/tools for to implement the test by implementing simple deployment test
[x] Prepare test cases
[x] Implements tests
[x] Scripts to prepare test environment
[x] CI/CD integration if applicable
[x] Update the documentation if applicable

hrishin commented 6 years ago

@rohanKanojia @piyush1594 Lets evaluate https://github.com/fabric8io/fabric8/tree/master/components/kubernetes-assertions for assertions

hrishin commented 6 years ago

To run these regression tests, first, we need to bring test environment in place

Running OpenShift (minishift) environment or Mock server (as per meeting discussion)
Script to bring this[1] environment on CI/CD instances To begin with the basic implementation, we would like to run the test against minishift then eventually substitute minishift with good mocking sever implementation.

@jstrachan @rohanKanojia @piyush1594 @rhuss WDYT?

hrishin commented 6 years ago

With the first run of regression tests, we need to fix the following issues

[x] Test behavior on Travis environment is intermittent. Sometimes tests are getting failed with some errors in f8-m-p like #1133 #1113 or may be issue with test infrastructure
[x] On an average CI job for test taking more than 20 minutes, travis log
[x] Fixed the build issue for OpenShift v3.7 #1130 as its affecting overall test behavior and performance as well

piyush-garg commented 6 years ago

@hrishin Can we close this now

piyush-garg commented 6 years ago

As #1198 is closed by #1199 and #1130 is also closed, I am merging this.