Accessing kubemci in k/k e2e tests

kubernetes / test-infra

Test infrastructure for the Kubernetes project.

Apache License 2.0

3.81k stars 2.62k forks source link

Accessing kubemci in k/k e2e tests #6624

Closed nikhiljindal closed 6 years ago

nikhiljindal commented 6 years ago

Context: Am adding an e2e test for kubemci as part of the existing ingress test suite: https://github.com/kubernetes/kubernetes/pull/59234. The test job needs to be able to run the kubemci binary.

How can I ensure that the job has access to kubemci binary?

@BenTheElder suggested that we can add a flag to kubetest to fetch the binary and make it available to the job. Filing this issue to make that happen.

More details: The binary is located at https://storage.googleapis.com/kubemci-release/release/v0.3.0/bin/linux/amd64/kubemci. I can create a new folder for latest instead of v0.3.0 that the test can use.

The tests are run as part of multiple jobs. https://k8s-testgrid.appspot.com/sig-network-gce#gci-gce-ingress is an example.

cc @BenTheElder @krzyzacy @G-Harmon @bowei @MrHohn

BenTheElder commented 6 years ago

/area kubetest /kind feature /assign /assign @krzyzacy

Want to get @krzyzacy to sign off on kubetest changes, but this seems reasonable and probably better than everyone maintining their own scripts to do this.

G-Harmon commented 6 years ago

Do we really want a flag for every binary added? That doesn't seem like it would scale well. Or are you saying the value of the flag is the data to fetch?

BenTheElder commented 6 years ago

Do we really want a flag for every binary added? That doesn't seem like it would scale well. Or are you saying the value of the flag is the data to fetch?

I would suggest we support obtaining a binary and placing it in the path based on the flag(s).

nikhiljindal commented 6 years ago

Yes good point @G-Harmon A generic flag should be enough here

krzyzacy commented 6 years ago

I'll question if that logic should live inside kubetest? The test probably should manage it's dependencies?

BenTheElder commented 6 years ago

@krzyzacy the test is e2e using ginkgo. kinda ugly to fetch binaries in the test and shove them in the path.

@nikhiljindal says they already publish binaries to GCS for consumption, so we can support fetching the latest binary from some flag

nikhiljindal commented 6 years ago

Created https://storage.googleapis.com/kubemci-release/release/latest/bin/linux/amd64/kubemci and pushed the binary there as well. Thats the URL we can use in our test to fetch the latest binary.

krzyzacy commented 6 years ago

ok, sounds reasonable

krzyzacy commented 6 years ago

discussed offline:

we can have another image similar to https://github.com/kubernetes/test-infra/blob/master/images/kubeadm/Dockerfile, which pulls latest kubemci image on top of latest kubekins-e2e
Set up a periodic job , similar to ci-ingress-gce-image-push, to 1) build/push latest kubemci binary to gcs bucket 2) push a new kubemci image, base off latest kubetest, with latest kubemci binary
In the e2e CI job, reference to kubemci:latest, which will always pull and test against latest kubemci

nikhiljindal commented 6 years ago

This is now done. We have a ci job that is pushing new images every time there is a commit: https://k8s-testgrid.appspot.com/sig-multicluster-kubemci#kubemci-image-push

https://k8s-testgrid.appspot.com/sig-multicluster-kubemci#kubemci-ingress-conformance is an example ci job that is running and using that pushed latest image in every run.