Go publish ignoring files causes checksum errors

[jackwhelpton]

When publishing a repo using a command such as:

rt gp go-dev-local $version --url=$HOST --user=$USER --password=$PASSWORD

certain files are not included in the archive uploaded to artifactory: the main culprit for me is .gitignore. This is causing some problems when pulling the package from artifactory, as it means the checksum no longer matches the value found in go.sum.

I think the code that does this may be:

// Returns a regex that match the following:
// 1. .git folder.
// 2. .gitignore file
// 3. .DS_Store
func getPathExclusionRegExp() (*regexp.Regexp, error) {
    excludePathsRegExp, err := regexp.Compile("(" + filepath.Join("^*", ".git", ".*$") + ")|(" + filepath.Join("^*", ".gitignore") + ")|(" + filepath.Join("^*", ".DS_Store") + ")")
    if err != nil {
        return nil, err
    }

    return excludePathsRegExp, nil
}

[AlexeiVainshtein]

Hi @jackwhelpton

Thank you for opening this issue. In the go.sum of the project, the checksums are of the dependencies and not the project itself. I couldn't reproduce this issue on my end. I created a package, published to Artifactory, different library consume that package from within Artifactory local repository and it worked. Can you please let us know where did you encountered the checksum issues? What go commands/Cli commands related to Go you run on the published package? Can you please share with us the step by step that you did so we could reproduce this issue on our end? Did the checksum error appeared on a different project that consume the published package? If so, did you used the replace block to point to the published project on your file system before publishing?

Thanks,

[jackwhelpton]

The problem is when pulling those dependencies from a project that references them, not on those packages directly.

Looking at my exact scenario the problem is manifesting itself when running go test, I wonder if that's required too. Let me see if I can produce a minimal test case, but it's something like this:

1. Publish dependency_1 by running JFROG_CLI_OFFER_CONFIG=false jfrog rt gp go-dev-local $version --url=$ARTIFACTORY --user=$USER --password=$PASSWORD from the directory containing its source
2. From a project that depends on dependency_1, run CGO_ENABLED=0 ./jfrog rt go 'get -d ./...' go-dev
3. Test this project by running CGO_ENABLED=0 ./jfrog rt go 'test ./... -coverprofile cover.out' go-dev | tee test.out

This results in something like this:

CGO_ENABLED=0 ./jfrog rt go 'test ./... -coverprofile cover.out' go-dev | tee test.out
[Info] Using go: go version go1.12.5 linux/amd64

go: downloading github.rakops.com/rcp/log/v4 v4.0.0
go: downloading github.rakops.com/rcp/container v0.2.1
go: downloading github.com/namsral/flag v1.7.4-pre
verifying github.rakops.com/rcp/container@v0.2.1: checksum mismatch
    downloaded: h1:IecPra9fDW6pMgPm9r0bc10ido473pVhSTw16XkPeAU=
    go.sum:     h1:tIxsaGGD1TNb5/6okGwfksv2oTmLgrdRguYdY+0Q3kk=
{
  "status": "failure",
  "totals": {
    "success": 0,
    "failure": 1
  }
}
[Error] exit status 1

[jackwhelpton]

I'm having trouble with a minimal test case myself, but I have noticed an interesting difference between these working test cases and the full failing scenario, which I'll include in the next comment. Here's what I've been doing that works:

Get the "hello" dependency

1. git clone https://github.com/jackwhelpton/hello.git

Create a new project that depends on "hello" for a test

2. mkdir hello_test && cd hello_test
3. echo -e 'package hello\n\nimport (\n\t"fmt"\n\t"testing"\n\n\t"github.com/jackwhelpton/hello"\n)\n\nfunc Test(t *testing.T) {\n\tfmt.Println(hello.Greeting("world"))\n}' > hello_test.go
4. echo -e 'module github.com/jackwhelpton/hello_test\n\ngo 1.12\n\nrequire github.com/jackwhelpton/hello v1.0.0' > go.mod
5. go test

This creates go.sum with:

  github.com/jackwhelpton/hello v1.0.0 h1:GTs9lpzk89CCEY+fhFpYWUvKGRIed0Q+N9k9TDu6jDo=
  github.com/jackwhelpton/hello v1.0.0/go.mod h1:E3p7SEfa0pS0dgWWgq1nn3oo5iwmclqj7Hsi7BQWlnk=

Now publish the hello dependency to artifactory:

6. cd ../hello && jfrog rt gp go-dev-local v1.0.0

Now let's simulate running this test on a clean build slave. Back in the hello_test directory, run:

7. cd ../hello_test && rm -rf $GOPATH/pkg/mod/github.com/jackwhelpton/hello@v1.0.0
8. jfrog rt go test go-dev

[jackwhelpton]

I now have two artifacts in artifactory, one of which (hello) is OK whereas the other (container) exhibits the checksum errors. When downloading the .zip files via Actions > Download on the directory containing all versions, the archives differ in an interesting way:

github.com
└── jackwhelpton
    └── hello@v1.0.0
        ├── .git
        ├── go.mod
        └── hello.go

github.rakops.com
└── rcp
    └── container@v0.2.1
        ├── container.go
        ├── containertest
        ├── container_test.go
        ├── contmysql
        ├── contredis
        ├── go.mod
        ├── go.sum
        ├── Jenkinsfile
        ├── Makefile
        └── README.md

Notice the inclusion of the .git directory in one case (the one that is working), and its omission in the other. I've limited the depth of the tree command here to keep the output readable: the .git directory has its own contents.

[AlexeiVainshtein]

Hi @jackwhelpton

Thank you for this information. We understand the issue and thinking about the solution regarding this. To better understand the issue, can you please elaborate regarding your use case? Can you please let us know why you perform the git clone and then jfrog rt gp on the dependency? This will help us better understand the issue and provide a suitable solution.

I also wanted to add that if you want to publish the dependency, you can do it without cloning the dependency. You can do it when you performjfrog rt go test <repository> --publish-deps=true and this will publish the dependencies of the project to the repository (If you are using CLI version below 1.25.0, you don't need to provide the flag). Also, if the dependency already in the cache, you can run the following command on the project itself that contains the dependency (in your case the hello_test project):jfrog rt gp <repository> <version> --deps=ALL --self=false. This will publish the dependencies of the project without publishing the project. In your case it will publish the hello dependency from the Go cache to Artifactory without publishing the hello_test project. In both the scenarios, the zip that was downloaded from Github with the .gitignore file will be deployed to Artifactory.

[jackwhelpton]

My use case is that I'm predominantly building CI/CD pipelines for an application and for its internal dependencies: the latter are shared across multiple applications and will be developed separately from any particular application, which is why you see them being published in isolation.

That said, the cloning and pushing of individual dependencies that I outlined above was an attempt to resolve (or at least reproduce) the checksum errors I was seeing rather than being part of my "usual" workflow.

For the main application, a typical pipeline looks something like this:

get: jfrog rt go --publish-deps "get -d $*" "$ARTIFACTORY_REPOSITORY" build: jfrog rt go --publish-deps "build -a -ldflags=\"-s -w -extldflags 'static'\" $*" "$ARTIFACTORY_REPOSITORY" test: jfrog rt go --publish-deps "test $*" "$ARTIFACTORY_REPOSITORY" | go2xunit -fail

is this a reasonable approach? I wonder if I should add --deps=ALL to these commands.

[jpreese]

@AlexeiVainshtein

FWIW I'm running into this issue as well, but I don't believe it to be .gitignore / git related. When I run a go build and then a go publish, my Artifactory instance says that I did not provide a checksum.

Exact error:

Client did not publish a checksum value.
If you trust the uploaded artifact you can accept the actual checksum by clicking the 'Fix Checksum' button.

And this appears to be similar, if not the same

https://www.jfrog.com/jira/browse/RTFACT-18446

The published .info .mod and .zip files all say Uploaded: None for both SHA-1 and MD5. SHA-256 does appear to have an uploaded checksum

This may also be the same for Conan

https://www.jfrog.com/jira/browse/RTFACT-15310

[nvx]

I ran into this same problem.

It looks like internally go uses the git archive command - https://github.com/golang/go/blob/1bca6cecc627cd708c6c5440eb14f84a99d5324b/src/cmd/go/internal/modfetch/codehost/git.go#L706

Creating a zip with this included the .gitignore file, while the jfrog-cli produced zip uploaded to Artifactory was missing the .gitignore file.

Naturally if a file is missing from the zip this would cause the SHA-256 to not match depending on if the dependency was fetched via Artifactory or if fetched directly from the git repo.

If there is some reason for this behaviour to persist, an option should be provided to control it so it matches the behaviour of Golang resolving artifacts from Git.

[eyalbe4]

@nvx, @jpreese and @jackwhelpton, Thank you all for providing your valuable input. Looks like there are two issues we need fix:

1. Make sure that the zip created is identical to the one created by github. We're working on fixing this now and will soon send out an update,.
2. Send the checksum headers to Artifactory when publishing a package. https://github.com/jfrog/jfrog-client-go/pull/78 handles this and will be released soon.

[nvx]

Did you mean to say identical to the one created by Go (which in turn leverages git archive)? The location inside the zip file in GitHub is different than what go modules expect.

A good way to compare against what Go does is to check the go mod cache folder after downloading modules without a proxy (or using the public proxy). For example $GOPATH/pkg/mod/cache/download/github.com/hashicorp/consul/@v/v1.5.1.zip which contains github.com/hashicorp/consul@v1.5.1/.gitignore - Note that the GitHub zip would have the .gitignore at the root level of the zip.

[eyalbe4]

Sure @nvx, Thanks for the tip - we'll compare the zip created by JFrog CLI with the one downloaded into the cache.

[jackwhelpton]

Any update on this? I've now got a particularly sticky module update whereby this prevents me building it with the proxy disabled (checksums will differ) but https://github.com/jfrog/jfrog-cli/issues/468 prevents me fetching dependencies with the proxy turned on.

[eyalbe4]

Sure @jackwhelpton - @Postyy created https://github.com/jfrog/jfrog-cli/pull/466 which fixes the checksum issue. We plan to merge it and get it released soon.

[jackwhelpton]

It's possible there's still an issue with this fix, but I may have applied it incorrectly. I grabbed the code, merged this branch and compiled a binary (I had to enable modules, so I think the documentation may be out of date, and also remove one replace that pointed to a non-existent dev version).

I used the resulting binary to publish a module that was causing me pain, and whilst it now includes the .gitignore it also seems to include the .git folder, which results in a checksum error for a different reason.

It's possible I built this incorrectly, and once it lands it'll just work, but can somebody else check this?

[Or-Geva]

@jackwhelpton, the .git folder is not included in the resulting binary. We will merge this fix ASAP.

[jackwhelpton]

Here's an interesting thing: I compiled the code with this fix using the Linux subsystem, pushed the package and the .git folder is correctly excluded (and .gitignore included). As before I had to delete a replace line to avoid:

go.mod:25: replace github.com/jfrog/gocmd: version "dev" invalid: unknown revision dev

Has this been tested on a Windows platform?

This isn't a critical problem for me as I have Jenkins responding to GitHub version creation and automatically pushing to Artifactory, so once I've updated those (Linux) slaves I should be good.

[jackwhelpton]

I've removed the contents of this post, as in the end I cut a new version of the troublesome module, which was enough to bust artifactory's cache. Without doing that I was repeatedly getting the old version of the module.

[jackwhelpton]

Another interesting one as I republish my dependencies:

$ jfrog rt gp go-dev v0.3.4
[Info] Using go: go version go1.13.1 linux/amd64

[Info] Publishing github.rakops.com/rcp/routing to go-dev
{
  "status": "failure",
  "totals": {
    "success": 0,
    "failure": 0
  }
}
[Error] open /mnt/c/Projects/github.rakops.com/rcp/routing/.gitignore: no such file or directory

This project has no .gitignore, but surely that shouldn't cause a failure in publishing?

[Or-Geva]

Thank you @jackwhelpton for your feedback. I have fixed the errors you mentioned, republish dependencies without '.gitignore' and go.mod:25: replace error.

[Or-Geva]

Dear @nvx, @jpreese and @jackwhelpton, this issue has been fixed (version 1.29.2), please update to the latest release in order to solve this.

[dnozay]

I am not sure this is fixed.

$ jfrog --version
jfrog version 1.33.2

$ jfrog rt go build main.go
[Info] Using go: go version go1.12.9 darwin/amd64

go: downloading github.com/fatih/color v1.9.0
verifying github.com/fatih/color@v1.9.0: checksum mismatch
    downloaded: h1:nHHU1ZN+sfpuT5HY2MMc2eOyCPq2z+PI/Wz2WB+7O10=
    go.sum:     h1:8xPHl4/q1VyqGIPif1F+1V3Y3lSmrq01EabUW3CoW5s=
exit status 1

main.go:

package main

import (
    "github.com/fatih/color"
)

func main() {
    c := color.New(color.FgCyan).Add(color.Underline)
    c.Println("Prints cyan text with an underline.")
}

I think it is still tripping on things with vendor files...

[jackwhelpton]

Confirmed: I see the same thing. It's preventing me building with dependencies pulled from Artifactory atm.