kubernetes-csi / csi-release-tools

shared build and test files used by kubernetes-csi projects
Apache License 2.0
21 stars 71 forks source link

Release armv7 images #167

Closed c0va23 closed 2 years ago

c0va23 commented 2 years ago

k8s has official builds for linux/arm platform, but csi tools do not have images for this platform. This small change fixes this.

Related issues: kubernetes-csi/node-driver-registrar#161.

k8s-ci-robot commented 2 years ago

Welcome @c0va23!

It looks like this is your first PR to kubernetes-csi/csi-release-tools 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-csi/csi-release-tools has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

k8s-ci-robot commented 2 years ago

Hi @c0va23. Thanks for your PR.

I'm waiting for a kubernetes-csi member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
mauriciopoppe commented 2 years ago

/ok-to-test

c0va23 commented 2 years ago

I'm investigating why the tests failed. At first glance, it looks like this is a compilation error due to 32 bits instead of 64.

c0va23 commented 2 years ago

/hold

c0va23 commented 2 years ago

After the https://github.com/kubernetes-csi/csi-driver-host-path assembly crashed, I decided to check if the rest of the projects would be assembled in the "kubernetes-csi" organization on a 32-bit architecture.

And I wrote a small script that clones all repositories, filters those that contain scripts from this project and tries to build projects for three architectures amd64, arm and 386.

./check_repos.sh

```bash #!/usr/bin/env bash # config ORG_NAME=kubernetes-csi EXCLUDE_REPO_NAMES="docs kubernetes-csi.github.io .github csi-proxy" ARCH_LIST="amd64 arm 386" # validate required executable curl --version jq --version git --version go version # load resos json if not loaded before if [[ -z "$repos_json" ]]; then echo "fetch repos_json" repos_json=$(curl "https://api.github.com/orgs/${ORG_NAME}/repos") fi repo_names=$(echo "$repos_json" | jq -r .[].name) # functions log() { local msg="$1" echo "> ${msg}" > /dev/stderr } include_repo() { local repo_name="$1" for exclude_repo_name in ${EXCLUDE_REPO_NAMES}; do if [[ "${exclude_repo_name}" = "${repo_name}" ]]; then return 1 fi done return 0 } clone_repo() { local repo_name="$1" if [[ -d "${repo_name}" ]]; then log "clone ${repo_name} exists" return 0 fi if git clone "git://github.com/${ORG_NAME}/${repo_name}.git"; then log "clone ${repo_name} success" return 0 else log "clone ${repo_name} failed" return 1 fi } run_in_dir() { local repo_name="$1" local command="$2" cd "${repo_name}" log "run ${command}" bash -c "${command}" run_status=$? cd .. return "$run_status" } is_suitable_repo() { local repo_name="$1" if [[ ! -f "${repo_name}/go.mod" ]]; then log "SKIP ${repo_name} no go " return 1 fi if [[ ! -d "${repo_name}/cmd" ]]; then log "SKIP ${repo_name} no cmd" return 1 fi if [[ ! -d "${repo_name}/release-tools" ]]; then log "SKIP ${repo_name} no release-tools " return 1 fi return 0 } build_cmd() { local repo_name="$1" local cmd_name="$2" local arch="$3" bin_name="${cmd_name}-${arch}" bin_path="${repo_name}/${bin_name}" build_command="GOARCH=${arch} go build -o ${bin_name} ./cmd/${cmd_name}" if run_in_dir "${repo_name}" "${build_command}"; then if [[ -f "${bin_path}" ]]; then log "SUCCESS: ${bin_path} binary exists" else log "ERROR: ${bin_path} not exists " fi else log "ERROR: build ${bin_path}" fi } build_repo() { local repo_name="$1" local cmd_names=$(ls "${repo_name}/cmd") for cmd_name in $cmd_names; do for arch in ${ARCH_LIST}; do build_cmd "${repo_name}" "${cmd_name}" "${arch}" done done } check_repos() { for repo_name in ${repo_names}; do echo # separator if ! include_repo "$repo_name"; then log "SKIP ${repo_name} excluded" continue fi if ! clone_repo "$repo_name"; then log "ERROR: ${repo_name} clone error" continue fi if ! is_suitable_repo "${repo_name}"; then continue fi build_repo "${repo_name}" done } ```

The rest of the projects were assembled without errors for all three architectures.

Script output

``` $ source check_repos.sh curl 7.78.0 (x86_64-pc-linux-gnu) libcurl/7.78.0 OpenSSL/1.1.1k zlib/1.2.11 zstd/1.5.0 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.44.0 Release-Date: 2021-07-21 Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd jq-1.6 git version 2.32.0 go version go1.16.2 linux/amd64 $ check_repos > clone csi-test exists > run GOARCH=amd64 go build -o csi-sanity-amd64 ./cmd/csi-sanity > SUCCESS: csi-test/csi-sanity-amd64 binary exists > run GOARCH=arm go build -o csi-sanity-arm ./cmd/csi-sanity > SUCCESS: csi-test/csi-sanity-arm binary exists > run GOARCH=386 go build -o csi-sanity-386 ./cmd/csi-sanity > SUCCESS: csi-test/csi-sanity-386 binary exists > run GOARCH=amd64 go build -o mock-driver-amd64 ./cmd/mock-driver > SUCCESS: csi-test/mock-driver-amd64 binary exists > run GOARCH=arm go build -o mock-driver-arm ./cmd/mock-driver > SUCCESS: csi-test/mock-driver-arm binary exists > run GOARCH=386 go build -o mock-driver-386 ./cmd/mock-driver > SUCCESS: csi-test/mock-driver-386 binary exists > clone external-attacher exists > run GOARCH=amd64 go build -o csi-attacher-amd64 ./cmd/csi-attacher > SUCCESS: external-attacher/csi-attacher-amd64 binary exists > run GOARCH=arm go build -o csi-attacher-arm ./cmd/csi-attacher > SUCCESS: external-attacher/csi-attacher-arm binary exists > run GOARCH=386 go build -o csi-attacher-386 ./cmd/csi-attacher > SUCCESS: external-attacher/csi-attacher-386 binary exists > clone external-provisioner exists > run GOARCH=amd64 go build -o csi-provisioner-amd64 ./cmd/csi-provisioner > SUCCESS: external-provisioner/csi-provisioner-amd64 binary exists > run GOARCH=arm go build -o csi-provisioner-arm ./cmd/csi-provisioner > SUCCESS: external-provisioner/csi-provisioner-arm binary exists > run GOARCH=386 go build -o csi-provisioner-386 ./cmd/csi-provisioner > SUCCESS: external-provisioner/csi-provisioner-386 binary exists > clone driver-registrar exists > SKIP driver-registrar no go > SKIP docs excluded > SKIP kubernetes-csi.github.io excluded > clone livenessprobe exists > run GOARCH=amd64 go build -o livenessprobe-amd64 ./cmd/livenessprobe > SUCCESS: livenessprobe/livenessprobe-amd64 binary exists > run GOARCH=arm go build -o livenessprobe-arm ./cmd/livenessprobe > SUCCESS: livenessprobe/livenessprobe-arm binary exists > run GOARCH=386 go build -o livenessprobe-386 ./cmd/livenessprobe > SUCCESS: livenessprobe/livenessprobe-386 binary exists > clone external-snapshotter exists > run GOARCH=amd64 go build -o csi-snapshotter-amd64 ./cmd/csi-snapshotter > SUCCESS: external-snapshotter/csi-snapshotter-amd64 binary exists > run GOARCH=arm go build -o csi-snapshotter-arm ./cmd/csi-snapshotter > SUCCESS: external-snapshotter/csi-snapshotter-arm binary exists > run GOARCH=386 go build -o csi-snapshotter-386 ./cmd/csi-snapshotter > SUCCESS: external-snapshotter/csi-snapshotter-386 binary exists > run GOARCH=amd64 go build -o snapshot-controller-amd64 ./cmd/snapshot-controller > SUCCESS: external-snapshotter/snapshot-controller-amd64 binary exists > run GOARCH=arm go build -o snapshot-controller-arm ./cmd/snapshot-controller > SUCCESS: external-snapshotter/snapshot-controller-arm binary exists > run GOARCH=386 go build -o snapshot-controller-386 ./cmd/snapshot-controller > SUCCESS: external-snapshotter/snapshot-controller-386 binary exists > run GOARCH=amd64 go build -o snapshot-validation-webhook-amd64 ./cmd/snapshot-validation-webhook > SUCCESS: external-snapshotter/snapshot-validation-webhook-amd64 binary exists > run GOARCH=arm go build -o snapshot-validation-webhook-arm ./cmd/snapshot-validation-webhook > SUCCESS: external-snapshotter/snapshot-validation-webhook-arm binary exists > run GOARCH=386 go build -o snapshot-validation-webhook-386 ./cmd/snapshot-validation-webhook > SUCCESS: external-snapshotter/snapshot-validation-webhook-386 binary exists > clone csi-driver-host-path exists > run GOARCH=amd64 go build -o hostpathplugin-amd64 ./cmd/hostpathplugin > SUCCESS: csi-driver-host-path/hostpathplugin-amd64 binary exists > run GOARCH=arm go build -o hostpathplugin-arm ./cmd/hostpathplugin # github.com/kubernetes-csi/csi-driver-host-path/pkg/hostpath pkg/hostpath/controllerserver.go:663:5: constant 4294967295 overflows int > ERROR: build csi-driver-host-path/hostpathplugin-arm > run GOARCH=386 go build -o hostpathplugin-386 ./cmd/hostpathplugin # github.com/kubernetes-csi/csi-driver-host-path/pkg/hostpath pkg/hostpath/controllerserver.go:663:5: constant 4294967295 overflows int > ERROR: build csi-driver-host-path/hostpathplugin-386 > clone csi-driver-iscsi exists > run GOARCH=amd64 go build -o iscsiplugin-amd64 ./cmd/iscsiplugin > SUCCESS: csi-driver-iscsi/iscsiplugin-amd64 binary exists > run GOARCH=arm go build -o iscsiplugin-arm ./cmd/iscsiplugin > SUCCESS: csi-driver-iscsi/iscsiplugin-arm binary exists > run GOARCH=386 go build -o iscsiplugin-386 ./cmd/iscsiplugin > SUCCESS: csi-driver-iscsi/iscsiplugin-386 binary exists > clone csi-driver-nfs exists > run GOARCH=amd64 go build -o nfsplugin-amd64 ./cmd/nfsplugin > SUCCESS: csi-driver-nfs/nfsplugin-amd64 binary exists > run GOARCH=arm go build -o nfsplugin-arm ./cmd/nfsplugin > SUCCESS: csi-driver-nfs/nfsplugin-arm binary exists > run GOARCH=386 go build -o nfsplugin-386 ./cmd/nfsplugin > SUCCESS: csi-driver-nfs/nfsplugin-386 binary exists > clone csi-lib-fc exists > SKIP csi-lib-fc no go > clone csi-lib-iscsi exists > SKIP csi-lib-iscsi no go > clone cluster-driver-registrar exists > SKIP cluster-driver-registrar no go > clone node-driver-registrar exists > run GOARCH=amd64 go build -o csi-node-driver-registrar-amd64 ./cmd/csi-node-driver-registrar > SUCCESS: node-driver-registrar/csi-node-driver-registrar-amd64 binary exists > run GOARCH=arm go build -o csi-node-driver-registrar-arm ./cmd/csi-node-driver-registrar > SUCCESS: node-driver-registrar/csi-node-driver-registrar-arm binary exists > run GOARCH=386 go build -o csi-node-driver-registrar-386 ./cmd/csi-node-driver-registrar > SUCCESS: node-driver-registrar/csi-node-driver-registrar-386 binary exists > clone csi-lib-utils exists > SKIP csi-lib-utils no cmd > clone external-resizer exists > run GOARCH=amd64 go build -o csi-resizer-amd64 ./cmd/csi-resizer > SUCCESS: external-resizer/csi-resizer-amd64 binary exists > run GOARCH=arm go build -o csi-resizer-arm ./cmd/csi-resizer > SUCCESS: external-resizer/csi-resizer-arm binary exists > run GOARCH=386 go build -o csi-resizer-386 ./cmd/csi-resizer > SUCCESS: external-resizer/csi-resizer-386 binary exists > clone csi-release-tools exists > SKIP csi-release-tools no go > clone csi-driver-image-populator exists > run GOARCH=amd64 go build -o imagepopulatorplugin-amd64 ./cmd/imagepopulatorplugin > SUCCESS: csi-driver-image-populator/imagepopulatorplugin-amd64 binary exists > run GOARCH=arm go build -o imagepopulatorplugin-arm ./cmd/imagepopulatorplugin > SUCCESS: csi-driver-image-populator/imagepopulatorplugin-arm binary exists > run GOARCH=386 go build -o imagepopulatorplugin-386 ./cmd/imagepopulatorplugin > SUCCESS: csi-driver-image-populator/imagepopulatorplugin-386 binary exists > SKIP csi-proxy excluded > clone external-health-monitor exists > run GOARCH=amd64 go build -o csi-external-health-monitor-agent-amd64 ./cmd/csi-external-health-monitor-agent > SUCCESS: external-health-monitor/csi-external-health-monitor-agent-amd64 binary exists > run GOARCH=arm go build -o csi-external-health-monitor-agent-arm ./cmd/csi-external-health-monitor-agent > SUCCESS: external-health-monitor/csi-external-health-monitor-agent-arm binary exists > run GOARCH=386 go build -o csi-external-health-monitor-agent-386 ./cmd/csi-external-health-monitor-agent > SUCCESS: external-health-monitor/csi-external-health-monitor-agent-386 binary exists > run GOARCH=amd64 go build -o csi-external-health-monitor-controller-amd64 ./cmd/csi-external-health-monitor-controller > SUCCESS: external-health-monitor/csi-external-health-monitor-controller-amd64 binary exists > run GOARCH=arm go build -o csi-external-health-monitor-controller-arm ./cmd/csi-external-health-monitor-controller > SUCCESS: external-health-monitor/csi-external-health-monitor-controller-arm binary exists > run GOARCH=386 go build -o csi-external-health-monitor-controller-386 ./cmd/csi-external-health-monitor-controller > SUCCESS: external-health-monitor/csi-external-health-monitor-controller-386 binary exists > clone csi-driver-smb exists > SKIP csi-driver-smb no cmd > SKIP .github excluded > clone lib-volume-populator exists > SKIP lib-volume-populator no cmd > clone volume-data-source-validator exists > run GOARCH=amd64 go build -o data-source-validator-amd64 ./cmd/data-source-validator > SUCCESS: volume-data-source-validator/data-source-validator-amd64 binary exists > run GOARCH=arm go build -o data-source-validator-arm ./cmd/data-source-validator > SUCCESS: volume-data-source-validator/data-source-validator-arm binary exists > run GOARCH=386 go build -o data-source-validator-386 ./cmd/data-source-validator > SUCCESS: volume-data-source-validator/data-source-validator-386 binary exists ```

c0va23 commented 2 years ago

/reopen

k8s-ci-robot commented 2 years ago

@c0va23: Reopened this PR.

In response to [this](https://github.com/kubernetes-csi/csi-release-tools/pull/167#issuecomment-897508602): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
c0va23 commented 2 years ago

/unhold

c0va23 commented 2 years ago

I removed the WIP label. This PR is ready for review.

mauriciopoppe commented 2 years ago

/lgtm /cc @pohly

k8s-ci-robot commented 2 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: c0va23, pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-csi/csi-release-tools/blob/master/OWNERS)~~ [pohly] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
c0va23 commented 2 years ago

/release-note-none