error prefetching hash: exit status 1

[lilyball]

I'm trying to use this on a project with a Gopkg.lock and when I run dep2nix I get the output

Found 22 projects to process.
* Processing: "[redacted]/common/golibs"
error prefetching hash: exit status 1

The project is an internal project but the import domain doesn't require authentication AFAICT (it just requires being on VPN, which I am). I don't know what "error prefetching hash" means. The Gopkg.lock file lists this for the given dependency:

[[projects]]
  digest = "1:8b59f86796b3cf384bde6c7b3ab2da9fa4c7eff057d216ccd420c80894a9c36f"
  name = "[redacted]/common/golibs"
  packages = [
    "bininfo",
    "pkgpath",
  ]
  pruneopts = "NUT"
  revision = "c3c7c6b379004afc72b5a25e220515c2da72d9e4"

[lilyball]

If it makes a difference, dep ensure works just fine.

[lilyball]

I'm still running into this. I really wish I could get this tool to at least print out the command it was executing.

[lilyball]

I just modified the source to print out the nix-prefetch-git args and the local path it's trying to prefetch from doesn't exist.

Edit: Actually it does exist while the tool is running, I guess it deletes the dir before returning.

[lilyball]

I figured out that stderr was being lost. After restoring that, I got the wonderful error

error: The path name 'git-ssh---git@redacted.example.com-common-golibs-c3c7c6b' is invalid: the '@' character is invalid. Path names are alphanumeric and can include the symbols +-._?= and must not begin with a period. Note: If 'git-ssh---git@redacted.example.com-common-golibs-c3c7c6b' is a source file and you cannot rename it on disk, builtins.path { name = ... } can be used to give it an alternative name.

[lilyball]

Ultimately, the problem is dep produces a git+ssh://git@redacted.example.com/common/golibs URL, which it checks out on disk as /tmp/935322867/sources/git-ssh---git@redacted.example.com-common-golibs-c3c7c6b, and nix-prefetch-git then takes the basename of that and tries to use that as the name in the nix store.

[lilyball]

This is rather hacky but seems to work:

diff --git a/prefetch.go b/prefetch.go
index dc3e74c..78b3b00 100644
--- a/prefetch.go
+++ b/prefetch.go
@@ -4,6 +4,9 @@ import (
    "bytes"
    "encoding/json"
    "github.com/Masterminds/vcs"
+   "io/ioutil"
+   "net/url"
+   "os"
    "os/exec"
    "strings"
 )
@@ -36,8 +40,31 @@ func cmdStdout(command string, arguments ...string) (string, error) {

 type gitPrefetcher struct{}

-func (p *gitPrefetcher) fetchHash(url string, revision string) (string, error) {
-   out, err := cmdStdout("nix-prefetch-git", "--url", url, "--rev", revision, "--quiet", "--fetch-submodules")
+func (p *gitPrefetcher) fetchHash(localUrl string, revision string) (string, error) {
+   if strings.Contains(localUrl, "@") {
+       // nix-prefetch-git does not sanitize the @ character, but it's not allowed in Nix paths.
+       // We need to copy this directory to another path that doesn't contain the @.
+       realUrl, err := url.Parse(localUrl)
+       if err != nil {
+           return "", err
+       }
+       dir, err := ioutil.TempDir("", "dep2nix")
+       if err != nil {
+           return "", err
+       }
+       defer os.RemoveAll(dir) // clean up the temp dir
+       // HACK: I don't want to write a recursive copy myself. I'll just leverage ditto instead.
+       cmd := exec.Command("ditto", "--", realUrl.EscapedPath(), dir)
+       cmd.Stderr = os.Stderr
+       if err := cmd.Run(); err != nil {
+           return "", err
+       }
+       localUrl = (&url.URL{
+           Scheme: "file",
+           Path:   dir,
+       }).String()
+   }
+   out, err := cmdStdout("nix-prefetch-git", "--url", localUrl, "--rev", revision, "--quiet", "--fetch-submodules")
    if err != nil {
        return "", err
    }

[lilyball]

It looks like we could use the --out flag to nix-prefetch-git instead, though I haven't tested that yet.