ricardosalveti
commented
3 years ago Thanks for reporting, will have a look.
Closed salerio-gs closed 3 years ago
ricardosalveti
commented
3 years ago Thanks for reporting, will have a look.
ricardosalveti
commented
3 years ago This issue seems to be related how go uses GOPROXY for fetching the modules that are dependencies for fioconfig. By default GOPROXY is set to “https://proxy.golang.org,direct”, which forces the google proxy as default option, and direct in case it is not found via proxy. In your case there seems to be a problem resolving proxy.golang.org, so that could be internal to your network.
The interesting part is that GOPROXY=direct would force it to use git directly, and that should make it work because git understands the http_proxy/https_proxy settings, but this doesn't work in OE because that logic is only part of the fetcher, and here the compile process is the one fetching all dependencies.
This seems to be a more generic golang issue in OE/Yocto (for projects using go mods), so will take a look at how we can fix this properly, but you can probably get it to work locally by changing the fioconfig recipe with the following:
meta-lmp$ git diff
diff --git a/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb b/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb
index fdb177e..4b7d65c 100644
--- a/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb
+++ b/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb
@@ -21,7 +21,7 @@ do_compile() {
BUILD_COMMIT=`git rev-parse --short HEAD`
FIOCONFIG_LDFLAGS="-X ${GO_IMPORT}/internal.Commit=${BUILD_COMMIT}"
mkdir -p ${B}/${GO_BUILD_BINDIR}
- ${GO} build -tags vpn -ldflags="${FIOCONFIG_LDFLAGS}" -o ${B}/${GO_BUILD_BINDIR}/fioconfig main.go
+ http_proxy="<your_proxy>" https_proxy="<your proxy>" GOPROXY=direct ${GO} build -tags vpn -ldflags="${FIOCONFIG_LDFLAGS}" -o ${B}/${GO_BUILD_BINDIR}/fioconfig main.go
chmod u+w -R ${B}
}
salerio-gs
commented
3 years ago Thanks for the detailed answer, I will give it a go. I did think our proxy might be not be resolving the default Go proxy, but I can get the offending module with wget, so it seems it might be an OE problem.
ricardosalveti
commented
3 years ago Thanks for the detailed answer, I will give it a go. I did think our proxy might be not be resolving the default Go proxy, but I can get the offending module with wget, so it seems it might be an OE problem.
Can you test building fioconfig outside OE?
git clone https://github.com/foundriesio/fioconfig.git
cd fioconfig
go build -o fioconfig main.goAssuming you got a recent go version installed at your host, it should also try to use the default settings for GOPROXY (and you can also set direct).
salerio-gs
commented
3 years ago I can build outside of OE, see below
peter@ubuntu:~/fioconfig$ go build -o fioconfig main.go
go: downloading github.com/urfave/cli v1.22.1
go: downloading github.com/doanac/crypto11 v1.2.2-0.20200715151421-f3d2e17ac497
go: downloading github.com/foundriesio/go-toml v1.8.1-0.20200721033514-2232fec316b9
go: downloading github.com/ethereum/go-ethereum v1.9.11
go: downloading github.com/urfave/cli/v2 v2.2.0
go: downloading github.com/miekg/pkcs11 v1.0.3-0.20190429190417-a667d056470f
go: downloading github.com/thales-e-security/pool v0.0.1
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d
go: downloading github.com/russross/blackfriday/v2 v2.0.1
go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0
peter@ubuntu:~/fioconfig$I also tried the patch to the recipe above and that works too :)
salerio-gs
commented
3 years ago Having got past the compilation phase I now have another QA issue. I don't know if this is related to changing the recipe.
ERROR: fioconfig-git-r0 do_package_qa: QA Issue: /usr/lib/go/pkg/mod/cache/vcs/dbb1616a13223a75321e21a8150b1a7781650b73e761213ecc0ab67568c38ac2/hooks/fsmonitor-watchman.sample contained in package fioconfig-staticdev requires /usr/bin/perl, but no providers found in RDEPENDS_fioconfig-staticdev? [file-rdeps]
ERROR: fioconfig-git-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
ERROR: Logfile of failure stored in: /build1/peter/PE2/LMP/build/tmp-lmp/work/cortexa72-lmp-linux/fioconfig/git-r0/temp/log.do_package_qa.114171
ERROR: Task (/build1/peter/PE2/LMP/build/conf/../../layers/meta-lmp/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb:do_package_qa) failed with exit code '1'
I also tried the patch to the recipe above and that works too :)
Great, so that is indeed a problem in OE (fetching during compile, which doesn't include the http_proxy variables).
Looking at how to fix this properly now.
ricardosalveti
commented
3 years ago Having got past the compilation phase I now have another QA issue. I don't know if this is related to changing the recipe.
ERROR: fioconfig-git-r0 do_package_qa: QA Issue: /usr/lib/go/pkg/mod/cache/vcs/dbb1616a13223a75321e21a8150b1a7781650b73e761213ecc0ab67568c38ac2/hooks/fsmonitor-watchman.sample contained in package fioconfig-staticdev requires /usr/bin/perl, but no providers found in RDEPENDS_fioconfig-staticdev? [file-rdeps] ERROR: fioconfig-git-r0 do_package_qa: QA run found fatal errors. Please consider fixing them. ERROR: Logfile of failure stored in: /build1/peter/PE2/LMP/build/tmp-lmp/work/cortexa72-lmp-linux/fioconfig/git-r0/temp/log.do_package_qa.114171 ERROR: Task (/build1/peter/PE2/LMP/build/conf/../../layers/meta-lmp/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb:do_package_qa) failed with exit code '1'
This one is interesting because in theory the same modules (and revs) would be used when downloading from scratch via proxy, so unclear what happened (maybe it got more source files to parse now?). Will check as well.
Adding the following should fix your QA warning:
RDEPENDS_${PN}-staticdev += "perl"
ricardosalveti
commented
3 years ago https://github.com/foundriesio/meta-lmp/commit/66329fc97b29accd4abbdabb0256a93ba440d34f and https://github.com/foundriesio/meta-lmp/commit/5be99f06bb00e73239d6c510347dc7871aa5c93a should fix this issue, testing now and will merge tomorrow if all fine during testing.
ricardosalveti
commented
3 years ago Changes merged on lmp-manifest, please sync to the latest on master and let me know if it fixes your issue.
salerio-gs
commented
3 years ago All good now, I'll leave you to close the issue?
I am seeing the compile failure below. As far as I know my http_proxy and htts_proxy environment variables are good and the proxy is working fine. If I edit the fioconfig recipe and insert GOPROXY=direct infront of the compilation command I get a similar error but only after a good couple of minutes of compilation. Any idea's what's wrong there?