search

cncf / cluster

🖥🖥🖥🖥CNCF Community Cluster
https://cncf.io/cluster
152 stars 42 forks source link

Vitess end to end testing port to Buildkite #184

Closed GuptaManan100 closed 2 years ago

GuptaManan100 commented 2 years ago

First and Last Name

Manan Gupta

Email

manan@planetscale.com

Company/Organization

PlanetScale

Job Title

Software Engineer

Project Title (i.e., a summary of what do you want to do, not what is the name of the open source project you're working with)

Vitess end to end testing porting to Buildkite

Briefly describe the project (i.e., what is the detail of what you're planning to do with these servers?)

For now Vitess is using GitHub Actions to run the CI but they are unreliable and require multiple re-runs. So we decided to port over to BuildKite.

Is the code that you’re going to run 100% open source? If so, what is the URL or URLs where it is located? What is your association with that project?

Yes, the project is Vitess and the pull request for the Proof of Concept is https://github.com/vitessio/vitess/pull/8556 I am a full time vitess engineer at PlanetScale.

What kind of machines and how many do you expect to use (see: https://metal.equinix.com/product/servers/)?

GitHub Actions provides us a concurrency of 60 actions at a time. For now, we only to want to test how many buildkite-agents we would be able to support on a c3.small and a c3.medium server. Once we have that information then we will be able to determine what our actual server requirement will be.

What operating system and networking are you planning to use?

Any OS would do that will be able to run docker-compose on it.

Any other relevant details we should know about?

If we do decide to go forward then we would also need to talk to BuildKite regarding the billing. They offer free plan for OSS projects https://buildkite.com/pricing Also, our end to end tests are CPU intensive and do not require much hard disk to run. Our PoC is hosted on a t2.medium AWS instance. Would it be possible to get AWS instances, since they would be much cheaper to run and would fulfill our requirements.

cc @deepthi

idvoretskyi commented 2 years ago

@GuptaManan100 I believe we can satisfy the server request, but before migrating to yet another CI tool, can you please explain in more details what your issues with GitHub Actions are?

GuptaManan100 commented 2 years ago

@idvoretskyi For us the GitHub Actions have become extremely flaky. A lot of the tests fail because they were unable to start a mysql instance. This makes us rerun a lot of the tests. This causes each PR to get stalled for long and it is painful for the developers. Also this bug seems to GitHub Action specific, unable to replicate in local environment. I didn't run into this issue in BuildKite either. Moreover, whenever GitHub Actions stop working, then all the PRs get stalled. So it seemed worthwhile to look for alternates for GitHub Actions

askdba commented 2 years ago

@idvoretskyi once we are able to estimate server needs, do you want me to create CNCF ticket?

GuptaManan100 commented 2 years ago

@idvoretskyi We have completed our experimentation and can now define our server needs. We request 2 c3.small servers which we would use for specific tests like vreplication. These servers would have to be running all the time.

idvoretskyi commented 2 years ago

@GuptaManan100 we've sent out invites to your team a week ago, so feel free to go ahead and create the desired instances.

I'll mark this request as closed.

GuptaManan100 commented 2 years ago

@idvoretskyi thankyou very much for your permission!