Couldn't read username for private repository

bazelbuild / bazel-gazelle

Gazelle is a Bazel build file generator for Bazel projects. It natively supports Go and protobuf, and it may be extended to support new languages and custom rule sets.

Apache License 2.0

1.17k stars 374 forks source link

Couldn't read username for private repository #768

Open stevenwjy opened 4 years ago

stevenwjy commented 4 years ago

What version of gazelle are you using?

v.0.20.0

What version of rules_go are you using?

v0.22.4

What version of Bazel are you using?

v3.0.0

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

MacOS 10.15.4

What did you do?

I tried to load a dependency from one of my private repositories (using git_repository).

I have tried using git config --global url."git@github.com:".insteadOf "https://github.com/" but it seems that the build process still tried to use http to access my repository. Besides, my access token is cached as well in my keychain, so previously when I only used bazel (without gazelle), I was able to fetch the repository perfectly fine. However, now that I want to use gazelle, this problem suddenly appeared.

What did you expect to see?

Successful build

What did you see instead?

fatal: could not read Username for 'https://github.com': terminal prompts disabled

stevenwjy commented 4 years ago

I tried running this on Ubuntu 18.04 as well, and same result.

I managed to run bazel build <path> where path refers to my go library that requires the private repo as a dependency. However, this problem occurred when using bazel run :gazelle.

Perhaps it is because gazelle uses another user to run the execution so it cannot see my ~/.gitconfig?

Btw, I managed to solve this issue by changing git_repository to go_repository. It seems bazel could understand how to build my code, but gazelle couldn't.

jayconrod commented 4 years ago

This is likely coming up due to Gazelle's dependency resolution. In short, Gazelle needs to resolve imports in .go files to the corresponding Bazel label. It first looks at library targets declared within your project (matching on importpath), then it looks for go_repository rules in WORKSPACE (again matching on importpath prefixes), then it will go out to the network using go list.

If you can use go_repository, I'd recommend that.

Otherwise, you can tell Gazelle to act as if a go_repository rule had been declared by adding a directive to WORKSPACE like this:

# gazelle:repo go_repository name=com_example_private importpath=example.com/private

That said, I'd expect Gazelle to find and use your ~/.gitconfig.

stevenwjy commented 4 years ago

Thanks for letting me know about that! I tried changing my git_repository to go_repository, and managed to run bazel run //:gazelle successfully.

However, when I tried using bazel run //:gazelle -- update-repos -from_file=go.mod, it failed with the same error fatal: could not read Username for 'https://github.com': terminal prompts disabled. Perhaps update-repos work in a different way?

jayconrod commented 4 years ago

update-repos -from_file=go.mod loads module information with go list -m all. It also can run git commands, depending on your GOPROXY setting.

Again, I'd expect it to find and use your ~/.gitconfig file though.

stevenwjy commented 4 years ago

Hmm... I'm not sure why but it does not seem to work in my case. I can successfully run bazel run //:gazelle and bazel build .... However, the problem still occurs with update-repos.

Here I tried to reproduce the error on my side, will really appreciate it if you have time to take a look: https://github.com/stevenwjy/bugtest/tree/master/bazelbuild-bazel-gazelle-issues-768

Thank you!

jayconrod commented 4 years ago

I couldn't reproduce this locally. Not sure if I followed quite the same steps.

In your repository, I changed github.com/stevenwjy/new-private-test to github.com/jayconrod/modtest (a scratch repository I made private)
I set GOPROXY=direct and GOSUMDB=off
I ran git config --global url."git@github.com:".insteadOf "https://github.com/"
- Incidentally, if there's a way to use OAuth or cookie based authentication, that would be much better. This requires SSH authentication, which for me, requires tapping a security key on every operation.
bazel run :gazelle -- update-repos -from_file=go.mod worked.
bazel query @com_github_jayconrod_modtest//... also worked.

So it looks like Gazelle, both in the main workspace and in go_repository is able to use ~/.gitconfig. I'm guessing there's another configuration issue here, either with Gazelle or the go command.

The prefix directive in your main build file is invalid by the way. It should be:

# gazelle:prefix github.com/stevenwjy/bugtest

stevenwjy commented 4 years ago

Ohh I see, I just noticed that the problem is with my go configuration. I had GOSUMDB="sum.golang.org", which was unable to verify the checksum for my private repository. I have managed to resolve this issue by setting GOPRIVATE accordingly.

Thank you very much for your help!

jayconrod commented 4 years ago

Cool, glad it's fixed.

Leaving this issue open, since Gazelle may need to do something with GOSUMDB. It's not clear to me that it's all that necessary for dependency resolution, but at the same time, we don't want to ingest modules into the cache without checking them.

antkillerfarm commented 2 years ago

I don't know this repo. But I meet the same problem on other repo which also use bazel. Maybe you can change repo url to fix it.

for example: git clone https://<USERNAME>:<PASSWORD>@github.com/path/to/repo1.git