RFE: Creation of a shared testing INFRA for CNCF

cncf / cluster

🖥🖥🖥🖥CNCF Community Cluster

https://cncf.io/cluster

154 stars 38 forks source link

RFE: Creation of a shared testing INFRA for CNCF #29

Closed timothysc closed 7 years ago

timothysc commented 7 years ago

I'm not entirely certain if this is the appropriate place to put this, but I believe it will leverage these resources.

The CNCF hosts a fair number of different projects which are controlled by a number of different vendors. The purpose of this request is to start the conversation about the establishment of a shared infrastructure (vendor neutral) area that can be used as a test bed for these projects. This ensures that releases have to go through a common set of shared resources that no-one vendor can explicitly control but they can all contribute to. ~= apache-infra

/cc @caniszczyk @spiffxp

caniszczyk commented 7 years ago

Thanks for filing this @timothysc, we've already been discussing this a bit and will get back to you soon on a tentative plan. We essentially have a start of this via the Intel cluster IMHO.

cc: @cncf/toc

spiffxp commented 7 years ago

https://groups.google.com/forum/#!topic/kubernetes-sig-testing/yFHu4LNu_y0

Some anecdata on the infra that google's currently using to test kubernetes

right now we have 91 VMs handling testing for PRs There are new builds of kubernetes roughly every hour. we have 40 VMs handling postsubmit testing. All our VMs have 8 cores Software-wise we depend on docker, golang and python.

Unclear to me whether this includes additional VM's spun up as part of cluster creation.

/cc @kubernetes/sig-testing-misc @fejta

timothysc commented 7 years ago

In the long haul I believe there needs to be more than the above requirements.

I think we should aim to push folks towards uniformity on the bottom of the stack, and that should not be a core differentiation unless it's to enable specific features.

We need to allow distro vendors to add their images and highlight capabilities and differences
- Ideally they would converge on some concensus
We need to have uniformity in the test-stack that lies atop
We need to create a standard deployment that is tried and tested, that only shifts when the community thinks it's wise.

If cncf projects are to be the shared kernel for the next generation of infrastructure, having a unified test bed is paramount to this effort. Otherwise, I fear we will be mired in churn on things that provide little value to end users, and typically leads to fragmentation.

/cc @jbeda

timothysc commented 7 years ago

@caniszczyk any news/update here?

dankohn commented 7 years ago

@timothysc We are beginning to discuss CNCF cross-project CI goals here: https://lists.cncf.io/pipermail/cncf-ci-public/2017-February/000001.html

I would encourage you to join that email list and make your suggestions. However, I want to suggest that it is extremely unlikely that CNCF would impose the same test bed on all CNCF projects, the way OpenStack has with Zuul. See https://lists.cncf.io/pipermail/cncf-ci-public/2017-February/000003.html

timothysc commented 7 years ago

Awesome, Thx @dankohn!

dankohn commented 7 years ago

Closing here, but please continue the discussion on the mailing list. Also, we're very happy to field specific, actionable requests for cluster allocations for CI here.