Support for go packages with multiple go.mod in subdirectories?

[dcermak]

I'm currently trying to package a go application, that uses go modules but whose go.mod lives in a subdirectory (actually it has multiple go.mod for a few subpackages, but I haven't got that far yet). I've looked at the code and it appears that a go.modin a subdirectory is currently unsupported.

Would you consider adding support for that? And maybe for multiple go.mod from the same archive too?

[jfkw]

Certainly, I'm actively looking for usecases like this. Can you provide some information on the directory layout and which Go packages in that tree you are attempting to build?

[dcermak]

The package in question is kubeflow: it has a plethora of go.mod in various subdirectories, which should probably be built into separate rpm packages. (Unfortunately it also has some code parts that don't use go modules, so that makes building "interesting")

[jfkw]

Documenting the Go module structure of kubeflow source here as a means of constructing a general use case for obs-service-go_modules:

Locations of go.mod files in the kubeflow source tree:

$ find . -name go.mod
./components/notebook-controller/go.mod
./components/gatekeeper/go.mod
./components/access-management/go.mod
./components/admission-webhook/go.mod
./components/profile-controller/go.mod
./bootstrap/go.mod

go build step in the Makefile:

%.so:
        cd cmd/plugins/$* && \
        ${GO} build -i -gcflags '-N -l' -o ../../../bin/$*.so -buildmode=plugin $*.go

All Directories in the source tree containing .go source files (gratuitous information):

./bootstrap/config/
./bootstrap/pkg/apis/
./bootstrap/pkg/apis/apps/
./bootstrap/pkg/apis/apps/kfdef/
./bootstrap/pkg/apis/apps/kfdef/v1alpha1/
./bootstrap/pkg/apis/apps/kfdef/v1alpha1/testdata/
./bootstrap/pkg/kfapp/
./bootstrap/pkg/kfapp/aws/
./bootstrap/pkg/kfapp/coordinator/
./bootstrap/pkg/kfapp/coordinator/fake/
./bootstrap/pkg/kfapp/dockerfordesktop/
./bootstrap/pkg/kfapp/existing_arrikto/
./bootstrap/pkg/kfapp/gcp/
./bootstrap/pkg/kfapp/gcp/fake/
./bootstrap/pkg/kfapp/gcp/testdata/
./bootstrap/pkg/kfapp/kustomize/
./bootstrap/pkg/kfapp/minikube/
./bootstrap/pkg/utils/
./bootstrap/version/
./components/access-management/
./components/access-management/kfam/
./components/access-management/pkg/apis/istiorbac/v1alpha1/
./components/access-management/pkg/apis/kubeflow/v1alpha1/
./components/admission-webhook/
./components/admission-webhook/pkg/apis/
./components/admission-webhook/pkg/apis/settings/
./components/admission-webhook/pkg/apis/settings/v1alpha1/
./components/gatekeeper/auth/
./components/gatekeeper/cmd/gatekeeper/
./components/gatekeeper/cmd/gatekeeper/options/
./components/notebook-controller/
./components/notebook-controller/api/v1alpha1/
./components/notebook-controller/controllers/
./components/notebook-controller/pkg/culler/
./components/notebook-controller/pkg/util/
./components/profile-controller/cmd/manager/
./components/profile-controller/pkg/apis/
./components/profile-controller/pkg/apis/istiorbac/v1alpha1/
./components/profile-controller/pkg/apis/kubeflow/
./components/profile-controller/pkg/apis/kubeflow/v1alpha1/
./components/profile-controller/pkg/controller/
./components/profile-controller/pkg/controller/profile/
./components/profile-controller/pkg/webhook/
./bootstrap/cmd/plugins/dockerfordesktop/
./bootstrap/cmd/kfctlClient/
./bootstrap/cmd/kfctl/
./bootstrap/cmd/kfctl/cmd/
./bootstrap/cmd/bootstrap/app/
./bootstrap/cmd/bootstrap/app/options/
./bootstrap/cmd/bootstrap/app/
./bootstrap/cmd/bootstrap/
./bootstrap/cmd/gc/

[e4t]

I've run into a similar issue packaging golang.org-x-tools-gopls which is a sub-project of golang.org-x-tools-gopls. Since this is contained within the sources of golang.org-x-tools-gopls as a subdirectory with its own go.mod, obs-service-go_modules presently is unable to handle it: it will find and tar up the dependencies in the top directory. Thus, an option to specify directories where to look for go.mod files would be useful. I'll be looking into this.

[alexandrevicenzi]

I have the same problem with etcd version 3.5.0. etcd provides multiple executables and each has its own go.mod. etcd project seems to encapsulate multiple executables/projects in one repository.

The current solution that I used was to add a vendor tarball for each executable manually, but I would like to have some automation.

One option is to add some parameters to go_modules, for example:

<service name="go_modules" mode="disabled">
   <param name="archive">etcd-3.5.0.tar.gz</param>
   <param name="output">vendor-etcd.tar.gz</param>
   <param name="gomod">server/go.mod</param>
</service>
<service name="go_modules" mode="disabled">
   <param name="archive">etcdctl-3.5.0.tar.gz</param>
   <param name="output">vendor-etcdctl.tar.gz</param>
   <param name="gomod">etcdctl/go.mod</param>
</service>

Would be nice to have an argument to tell where the go.mod lives and what the output name should be.

Another option is to automagically find all go.mod files and create multiple folders inside vendor.tar.gz, for example:

vendor.tar.gz
├── etcdctl/vendor/*
├── etcdctl/vendor/go.mod
├── server/vendor/*
└── server/vendor/go.mod

This way I can unpack the vendor folder in the desired location.

I'm willing to implement one of these solutions and send a PR.