go module workspaces

[urandom2]

Describe the bug

while attempting to package atlas for #197774, it became apparent that buildGoModule is not capable of handling the new go module workspace mode.

technically the versions of go we have supports it, but because workspaces contradict -mod=vendor, see golang/go#54130, we may need to add buildGoWorkspace?

Steps To Reproduce

Steps to reproduce the behavior:

1. have a repo with go.mod and go.work
2. use buildGoModule

   go: -mod may only be set to readonly when in workspace mode, but it is set to "vendor"
   Remove the -mod flag to use the default readonly value,
   or set GOWORK=off to disable workspace mode.

Expected behavior

derivation builds correctly

Additional context

since we are using GOPROXY=file:// to inject dependencies, we should not need to use -mod=vendor.

Notify maintainers

@c00w @kalbasit @mic92 @zowoq

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.79, NixOS, 22.11 (Raccoon), 22.11.20221121.af50806`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(root): `""`
 - channels(Colin Arnott): `"nixos-22.05-22.05"`
 - channels(colin): `""`
 - nixpkgs: `/nix/store/lwnrrndriqm2d588lgd6184bcawj3b0z-source`

[zowoq]

since we are using GOPROXY=file:// to inject dependencies, we should not need to use -mod=vendor.

Don't understand what you mean?

https://github.com/NixOS/nixpkgs/blob/eaf80fc85ec93b20594dfc186df6a63dff07abc9/pkgs/build-support/go/module.nix#L168

https://github.com/NixOS/nixpkgs/blob/eaf80fc85ec93b20594dfc186df6a63dff07abc9/pkgs/build-support/go/module.nix#L180-L181

[urandom2]

error: builder for '/nix/store/6mx0zawbpkdrlsl8y1y5v33lm2dzmmz5-atlas-0.8.0.drv' failed with exit code 1;
       last 10 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/alkc8id9xjs3aas1jm2qfskcfkywcxhk-source
       > source root is source
       > patching sources
       > configuring
       > building
       > Building subPackage .
       > go: -mod may only be set to readonly when in workspace mode, but it is set to "vendor"
       >     Remove the -mod flag to use the default readonly value,
       >        or set GOWORK=off to disable workspace mode.
       For full logs, run 'nix log /nix/store/6mx0zawbpkdrlsl8y1y5v33lm2dzmmz5-atlas-0.8.0.drv'.
error: 1 dependencies of derivation '/nix/store/9fj79dfz1i8pplpg9h2ym4ksr9p0yxhh-review-shell.drv' failed to build

https://github.com/NixOS/nixpkgs/compare/master...urandom2:nixpkgs:atlas

EDIT: let me know what additional context you are curious about?

[zowoq]

https://github.com/NixOS/nixpkgs/compare/master...urandom2:nixpkgs:atlas

That isn't using GOPROXY=file:// ?

[zowoq]

atlas has an open PR that builds: https://github.com/NixOS/nixpkgs/pull/201531

[urandom2]

sorry, misread the source; but you piqued my interest and proxyVendor = true; made it work. should we document this better?

meta question, why is proxyVendor ? true not default?

[MatrixManAtYrService]

I think I'm seeing this bug (here's the flake with the problem: https://gist.github.com/MatrixManAtYrService/ac040f60d3602fc2df871623b1d09bf7).

In my case, adding proxyVendor = true; resulted in a different error:

../kyaml/openapi/openapi.go:22:2: github.com/evanphx/json-patch@v4.11.0+incompatible: reading file:///nix/store/nx9lf6v3pbbv1140i112xvzr818pmz3z-kustomize-4.5.4-go-modules/github.com/evanphx/json-patch/@v/v4.11.0+incompatible.mod: no such file or directory

I'm posting 90% because I want to help collect data that helps with fixing bugs and 10% because maybe sombody can tell me how to work around it in my flake :wink: . Thanks all.

[jalaziz]

UPDATE: I got it working using this tip to force regenerating the vendor has.

Ran into this myself while trying to upgrade a package :(

Unfortunately, the export GOWORK=off simply resulted in a new issue.

Given this:

{ lib, buildGoModule, fetchFromGitHub, go-mockery, runCommand, go }:

buildGoModule rec {
  pname = "go-mockery";
  version = "2.32.0";

  src = fetchFromGitHub {
    owner = "vektra";
    repo = "mockery";
    rev = "v${version}";
    sha256 = "sha256-fQzXgCRMIcGQRCnKn/vu3GzNrx4/xrMVmzqjOujyNNE=";
  };

  preCheck = ''
    substituteInPlace ./pkg/generator_test.go --replace 0.0.0-dev ${version}
  '';

  preBuild = ''
    export GOWORK=off
  '';

  ldflags = [
    "-s" "-w"
    "-X" "github.com/vektra/mockery/v2/pkg/config.SemVer=v${version}"
  ];

  CGO_ENABLED = false;

  vendorHash = "sha256-ALcozrbP8vj3CEZkUVpeVPdi2ok89F9qK62xaVf2E0c=";

  passthru.tests = {
    generateMock = runCommand "${pname}-test" {
      nativeBuildInputs = [ go-mockery ];
      buildInputs = [ go ];
    } ''
      if [[ $(mockery --version) != *"${version}"* ]]; then
        echo "Error: program version does not match package version"
        exit 1
      fi

      export HOME=$TMPDIR

      cat <<EOF > foo.go
      package main

      type Foo interface {
        Bark() string
      }
      EOF

      mockery --name Foo --dir .

      if [[ ! -f "mocks/Foo.go" ]]; then
        echo "Error: mocks/Foo.go was not generated by ${pname}"
        exit 1
      fi

      touch $out
    '';
  };

  meta = with lib; {
    homepage = "https://github.com/vektra/mockery";
    description = "A mock code autogenerator for Golang";
    maintainers = with maintainers; [ fbrs ];
    mainProgram = "mockery";
    license = licenses.bsd3;
  };
}

The build fails with:

main module (github.com/vektra/mockery/v2) does not contain package github.com/vektra/mockery/v2/tools

tools here is a separate module in the same workspace, which is properly handled by Go workspaces.

If I remove the GOWORK=off workaround, I end up with all the issues mentioned in this ticket. Not sure what else to do.

[katexochen]

I'm not sure we should add a builder for Go workspaces, but in case we decide to do so, there is an interesting change in Go 1.22 that allows to vendor dependencies of a Go workspace:

Commands in workspaces can now use a vendor directory containing the dependencies of the workspace. The directory is created by go work vendor, and used by build commands when the -mod flag is set to vendor, which is the default when a workspace vendor directory is present.

Note that the vendor directory's contents for a workspace are different from those of a single module: if the directory at the root of a workspace also contains one of the modules in the workspace, its vendor directory can contain the dependencies of either the workspace or of the module, but not both.

https://tip.golang.org/doc/go1.22#go-command

[katexochen]

for cross-reference: https://github.com/NixOS/nixpkgs/issues/299096