Run tests on GKE spot instances

knative / infra

Home of Infra (Productivity) that hosts configs for prow and other infrastructure related things.

Apache License 2.0

8 stars 27 forks source link

Run tests on GKE spot instances #384

Open cardil opened 8 months ago

cardil commented 8 months ago

GKE provides, so-called spot instances. The main benefit, of using them, is a massive cost savings. The GKE with Spot instances doc describes how to use them.

Although, it would be a good idea to run tests on those nodes, some changes are probably needed in Knative:

[ ] ensure the tests could survive node termination
[ ] reconfigure test clusters to use more, but smaller nodes

jasonsmithio commented 5 months ago

/assign

jasonsmithio commented 5 months ago

A question with regards to this. What are the outcomes we are looking for in this test? I am happy to run it but want to know what would be considered a success wrt this issue.

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen the issue with /reopen. Mark the issue as fresh by adding the comment /remove-lifecycle stale.

cardil commented 1 month ago

/remove-lifecycle stale

@jasonsmithio Thanks, you wanna tackle this one. However, It is not a matter of writing some new tests for spot instances. The current Knative e2e tests have to work on a test cluster made of spot instances (the nodes of the cluster could be evicted at any time). We need to make sure that the tests do not produce false negatives due to random node evictions.

jasonsmithio commented 1 month ago

gotcha. It may take a while to test as we need the nodes to evict and do it enough times to collect enough data.