Could not compile v0.38.0 on air-gaped environment

[psycofdj]

Bug Report Info

Currently:

The v0.38.0 release cannot compile on air-gaped environment as many go packages fetch their dependencies on internet. Example for package migrate-to-pxc provided in reference section.

I believe this new behaviour was introduced by commit 1c6ff903fa458b197e5ceb6905238924704ba00f (between releases v0.37.0 and v0.38.0).

Expected:

We expect the release to somehow compile on air-gaped environments.

Is it possible to configure proxy environment variables on bosh compilation VMs ?

Reference:

Package migrate-to-pxc compilation error:

Task 34252 | 06:09:40 | Compiling packages: migrate-to-pxc/61ced9712075ab812f974e0262a85593091ceb064ddd8a9c39b62faea4ab7791 (00:03:06)
                     L Error: Action Failed get_task: Task dca53e5d-0d4f-4fdf-437c-5ba0d958b7de result: Compiling package migrate-to-pxc: Running packaging script: Running packaging script: Command exited with 1; Stdout: , Stderr: + source /var/vcap/packages/golang-1.16-linux/bosh/compile.env
++ '[' -z '' ']'
+++ readlink -nf /var/vcap/packages/golang-1.16-linux
++ export GOROOT=/var/vcap/data/packages/golang-1.16-linux/8bc1c6148323cc0a24892cdb3cc2b443a1f96810
++ GOROOT=/var/vcap/data/packages/golang-1.16-linux/8bc1c6148323cc0a24892cdb3cc2b443a1f96810
++ export GOPATH=/var/vcap/data/compile/migrate-to-pxc
++ GOPATH=/var/vcap/data/compile/migrate-to-pxc
++ export GOCACHE=/var/vcap/data/golang-1.16-linux/cache
++ GOCACHE=/var/vcap/data/golang-1.16-linux/cache
++ export PATH=/var/vcap/data/packages/golang-1.16-linux/8bc1c6148323cc0a24892cdb3cc2b443a1f96810/bin:/var/vcap/data/compile/migrate-to-pxc/bin:/var/vcap/bosh/bin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin
++ PATH=/var/vcap/data/packages/golang-1.16-linux/8bc1c6148323cc0a24892cdb3cc2b443a1f96810/bin:/var/vcap/data/compile/migrate-to-pxc/bin:/var/vcap/bosh/bin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin
+ export GOBIN=/var/vcap/packages/migrate-to-pxc/bin
+ GOBIN=/var/vcap/packages/migrate-to-pxc/bin
+ cd migrate-to-pxc
+ go install github.com/cloudfoundry/migrate-to-pxc
go: github.com/cloudfoundry/gosigar@v1.2.0: Get "https://proxy.golang.org/github.com/cloudfoundry/gosigar/@v/v1.2.0.mod": dial tcp 172.217.18.241:443: i/o timeout

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/179732578

The labels on this github issue will be updated when the story is started.

[abg]

Confirmed. This behavior change was introduced in v0.38.0 unintentionally as a side effect of removing vendored Go dependencies.

Unfortunately, as far as I am aware there is no way to plumb in proxy environment variables into the bosh compilation VMs or otherwise provide "deploy time" configuration to the compilation environment.

To support this use case, I think go vendoring will need to be added back to the relevant components and support added back to the packaging scripts to consume that vendoring.

[abg]

This has been fixed in the v0.39.0 release. This should be available on bosh.io shortly.

We have adjusted our CI testing pipeline to avoid this sort of regression in the future by always compiling pxc-release in a BOSH environment whose compilation VMs do not have public internet access.

[psycofdj]

v0.39.0 compiles fines, thank you for your work !