CI flows on branches should use stable releases

knative / test-infra

Test infrastructure for the Knative project

Apache License 2.0

82 stars 161 forks source link

CI flows on branches should use stable releases #721

Closed adrcunha closed 5 years ago

adrcunha commented 5 years ago

Currently CI flows on branches use nightly releases. For example,

https://gubernator.knative.dev/build/knative-prow/logs/ci-knative-eventing-0.5-continuous/1123145177083416576

fails when installing Istio from the latest Knative Serving nightly. The flow should be using artifacts from Knative Serving 0.5.

@mattmoor @tcnghia to validate the assumption.

adrcunha commented 5 years ago

/assign @dushyanthsc

dushyanthsc commented 5 years ago

/assign @Fredy-Z

dushyanthsc commented 5 years ago

/unassign @dushyanthsc

chizhg commented 5 years ago

Eventing has completely got rid of Serving and Istio, and it's not calling start_latest_knative_serving in e2e-tests.sh anymore. So for Eventing 0.5, we can just create a new function start_knative_serving_0.5, and use it to replace start_latest_knative_serving.

But Eventing-sources (the future Eventing-contrib) will still be relying on the latest Eventing. How do we make changes and get it synced when we cut the new release? Can we make the configuration external and modify it in the auto-release process?

adrcunha commented 5 years ago

This issue requires a general solution, for all repos, not only eventing or eventing-sources. This should be done in the helper script, then propagated to the other repos.

adrcunha commented 5 years ago

0.6 will be cut next week, we should have this fixed by then to have a clean release branch CI.

chizhg commented 5 years ago

I have some thoughts on how to solve this issue, but not sure if it is feasible or the best solution.

Modify make_config.go to add a VERSION_NUMBER environment variable for configuration of prow jobs that run for released branches.

Change the current functions like start_latest_knative_serving to start_knative_serving. Basically the function will be like:

function start_knative_serving() {
version = ${VERSION_NUMBER}
if version == "" {
// start latest knative serving
} else if version == "0.5" {
// start istio and knative serving 0.5
} else {
// start knative serving of the given version
}
}

adrcunha commented 5 years ago

I can simply check the current branch. Release branches don't have presubmits, and CI flows perform a standard clone/checkout for running a test.

The right fix is not changing start_knative_serving() but KNATIVE_BASE_YAML_SOURCE: https://github.com/knative/test-infra/blob/master/scripts/library.sh#L26

chizhg commented 5 years ago

Can you elaborate more on this? Seems merely changing KNATIVE_BASE_YAML_SOURCE is not enough. The URL for released yaml files is like https://github.com/knative/serving/releases/download/v0.5.0/serving.yaml, which is quite different from the nightly-release. And the installation steps might be different from version to version, for example, eventing<=0.5 still needs serving but eventing>0.5 does not.

Also do we need to consider https://github.com/knative/test-infra/issues/675 as well in this issue?

adrcunha commented 5 years ago

675 will probably require a different approach. Regarding the URL, point to the knative-releases GCS, not to GitHub, e.g., https://storage.googleapis.com/knative-releases/serving/previous/v0.5.0/release.yaml

chizhg commented 5 years ago

675 will probably require a different approach. Regarding the URL, point to the knative-releases GCS, not to GitHub, e.g., https://storage.googleapis.com/knative-releases/serving/previous/v0.5.0/release.yaml

Is this URL wrong? It shows No such object: knative-releases/serving/previous/v0.5.0/release.yaml.

adrcunha commented 5 years ago

Indeed. The correct URL is https://storage.googleapis.com/knative-releases/serving/previous/v0.5.0/serving.yaml