search

Peripli / service-manager

Service Manager Core
Apache License 2.0
69 stars 28 forks source link

Nir reduce travis execution time #520

Closed nirbenrey closed 4 years ago

nirbenrey commented 4 years ago

Motivation

Running time of travis job is over 50 minutes. After 50 minutes the job fails because it is the max execution time for travis free plan. In addition it is time consuming. We would like to reduce the execution time to 15 minutes.

Approach

In order to reduce the execution time we split the test execution in travis into 5 separate stages that run concurrently. Each stage is self contained and can be re-triggered from travis UI. The stages are:

  1. Broker integration tests (make goal - precommit-integration-tests-broker)
  2. OSB and Plugin Integration Tests (make goal - precommit-integration-tests-osb-and-plugin)
  3. Service Instance and Service Bindings Integration Tests (make goal - precommit-integration-tests-service-instance-and-binding)
  4. Other Integration Tests (make goal - precommit-integration-tests-other)
  5. Unit tests (make goal - precommit-unit-tests)

Each stage triggers different goal in the make file that runs different tests and generate coverage report. Then By using coveralls flag "COVERALLS_PARALLEL=true" and coveralls webhook we merge the reports.

coveralls commented 4 years ago

Coverage Status

Coverage increased (+0.1%) to 86.504% when pulling 7a6e6040576407374f385a46dd2a404c5cea3ac8 on nir--reduce-travis-execution-time into e0504da151143acdaff2f5b6d0f0b63906cf4323 on master.