search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.13k stars 3.81k forks source link

testing: integrate binary verification into TeamCity #19238

Open tbg opened 7 years ago

tbg commented 7 years ago

The release process has a step which verifies the binaries but there's currently an option between a fairly shallow manual check or the more extensive bincheck courtesy of @benesch.

In light of https://github.com/cockroachdb/cockroach/pull/19225 I wanted to add a check that makes sure that the cluster version matches the binary version, but this isn't going to be effective unless we make bincheck our primary method of verification.

Is that a reasonable course of action?

Epic RE-227 Jira issue: CRDB-5985

richardwu commented 7 years ago

+1

lgo commented 7 years ago

There's a lot of value in having CI run these checks. To name a few, preventing mistakes, catching any regressions of compatibility with other platforms, and for extending the verification process to include more tests. Maybe even logic tests and acceptance tests.

Having these checks also be done on CI isn't any worse than already relying on CI to build the binaries.

benesch commented 7 years ago

Yes! I've wanted to move bincheck into TeamCity for a while. The blocker has been getting Mac and Windows build agents; previously I was (ab)using Travis and Appveyor for this.

We could always start with the Linux verification—that's the most important one, anyway—and work our way up from there.

lgo commented 7 years ago

I notice you've since moved bincheck to the cockroachdb org and removed the manual check notes from the release process. Seems like this should be closed?

I do think we should create an issue to improve bincheck, for example, run a cluster version check @tschottdorf mentioned. Maybe something like running logictests and acceptance tests against the binary? (helpful for ensuring cross-platform works rather than a sanity check)

benesch commented 7 years ago

I say we leave it open to track making TeamCity run bincheck automatically.

github-actions[bot] commented 3 years ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 5 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!

rail commented 3 years ago

Reopening this. We still want this to happen.

kenliu-crl commented 2 years ago

note for future reference: we want to run this check earlier in the build process than at the end right before a release