openconfig / models-ci

Continuous integration for OpenConfig models.
Apache License 2.0
1 stars 0 forks source link

Use go modules for misc-checks and goyang/ygot #59

Open wenovus opened 3 years ago

wenovus commented 3 years ago

It looks like go modules would've prevented these failures from happening (the third failure): https://github.com/openconfig/models/pull/1016

wenovus commented 3 years ago

I actually can't figure out the root cause of this anymore :(, but since it's a rare and low-impact issue anyhow, I think we can just wait for the next occurrence.

wenovus commented 2 years ago

This issue recently appeared when goyang was given a backwards-incompatible change: https://github.com/openconfig/public/pull/559/commits/347a408defdafa29f6f32905a6601e233d915c4b

While not verified, I believe the cause is due to the generated code being placed in its own repository during the CI process, such that when go get is called, it doesn't use the dependencies of the latest ygot, but rather the latest versions of all of the dependencies of the generated code. Since ygot hasn't been updated to the latest version of goyang, the backwards incompatibility surfaces as a build error.

One way to solve this problem is to somehow get a repository of the latest version of ygot into the GOPATH, and then place the generated code there before building so that it's not in its own repo.

Here is a diff block that can get this started:

diff --git a/cloudbuild.yaml b/cloudbuild.yaml
index e22be21a..8926b4b9 100644
--- a/cloudbuild.yaml
+++ b/cloudbuild.yaml
@@ -242,6 +242,23 @@ steps:
   - 'GO111MODULE=on'
   waitFor: ['go path creation']
   id: 'goyang-ygot prep'
+# Clone CI repository
+- name: 'gcr.io/cloud-builders/go:debian'
+  entrypoint: 'bash'
+  args:
+  - '-c'
+  - |
+    git clone git@github.com:openconfig/ygot.git /go/src/github.com/openconfig/ygot
+    cd /go/src/github.com/openconfig/ygot
+    git fetch --tags
+    git checkout $(git describe --tags $(git rev-list --tags --max-count=1))
+  volumes:
+  - name: 'ssh'
+    path: /root/.ssh
+  - name: 'gopath'
+    path: /go
+  waitFor: ['go path creation']
+  id: 'ygot clone'
 - name: 'gcr.io/$PROJECT_ID/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', "/go/src/github.com/openconfig/models-ci/validators/goyang-ygot/test.sh"]
@@ -256,7 +273,7 @@ steps:
   - 'COMMIT_SHA=$COMMIT_SHA'
   - '_REPO_SLUG=$_REPO_SLUG'
   - 'BRANCH_NAME=$BRANCH_NAME'
-  waitFor: ['validator prep', 'goyang-ygot prep', 'oc-pyang']
+  waitFor: ['validator prep', 'goyang-ygot prep', 'ygot clone', 'oc-pyang']
   id: 'goyang-ygot'

 ############### PYANG ###############