Support for the Go language

[dgpc]

Since the Roadmap mentions this under "Tentative post-beta plans", filing this issue so folks who are interested can subscribe for updates.

[ulfjack]

Han-Wen has a design proposal. We need the new rules to be (at least partially) compatible with our internal rules as we have cases where people use both the internal and external tools for the same code.

[hanwen]

Plan was ok'd by the Go team.

[mzhaom]

Any timeline about when this would happen?

[damienmg]

I hope we can do it by the end of the summer but we don't have a schedule yet. We will update the roadmap with a more precise plan for the post-beta when the beta is out.

[hanwen]

rough sketch at https://docs.google.com/document/d/1_fNIFC-2gLETPBXWipNDaZEi1NCKDJRBqGN7lF0Eo7k/edit#

[ghost]

I have noticed Bazel added Rust lang support. That's not a good news for golang. Bazel already support Rus tlang, Golang not , Gophers crying.... Why not eat our own dog food. Hope you will support golang first.

Golang is your programming language. May we put it at the first class support stage family?

The next battle is about programming. Now we hope can using golang code for more things. webapp, system software, tools. android apps, ios apps.

Not a good building tools, Things will be harder to do.

[kchodorow]

We are planning to add go support in the next couple of months, check out the updated roadmap: http://bazel.io/roadmap.html.

[pspeter3]

Does anyone have Skylark rules for Go in the interim?

[damienmg]

Hanwen's prototype is here: https://github.com/hanwen/examples/commit/8cb9da8c31a078168f23b01ce146ffe7cf7c5302

[pspeter3]

Is there a suggest way to structure the project for Go with Bazel?

[damienmg]

https://github.com/hanwen/examples/blob/8cb9da8c31a078168f23b01ce146ffe7cf7c5302/tools/go/README.md

In the README Han-wen's said the workspace is expected to live in $GOROOT/src/github.com/joe/project/ and be compatible with the go tooling. We expect to support '.' in package name to make our go rules 100% compatible with the go tooling.

[pspeter3]

Thanks!

[thefallentree]

$GOROOT here means WORKSPACE root , right?

[hanwen]

No. The GOROOT usually contains packages from other sources too.

you would have

 $GOROOT/src/github.com/joe/project/WORKSPACE
 =>
 sets up toolchain

$GOROOT/src/github.com/joe/project/BUILD
 =>
 go_prefix("github.com/joe/project")

On Thu, Oct 8, 2015 at 3:44 PM, Yucong Sun notifications@github.com wrote:

$GOROOT here means WORKSPACE root , right?

— Reply to this email directly or view it on GitHub https://github.com/bazelbuild/bazel/issues/79#issuecomment-146549784.

Han-Wen Nienhuys Google Munich hanwen@google.com

[thefallentree]

Well, that's not very inconvenient and pretty confusing , at least to me . a workspace is usually considered to be a all-inclusive directory that contains everything a build process would take. There will be multiple workspace in various location on anyplace and I can build everything out of that directory.

I don't understand the purpose to support standard go tool in bazel . It's not like we support using maven or modifying pom.xml for java support. What is the main purpose to support go tool? IMHO, just support a vendor directory, and ask people to use whatever they feel necessary to dump source code there. (or support a way to fetch that on build time, like java rule does).

Trying to mimic go tool only cause pain, i think.

[hanwen]

It was a specific request from the Go team for the Bazel rules to interoperate with the 'go' tool.

In the case of Bazel, other directories under $GOROOT will be ignored.

[hanwen]

also, to build something with bazel, you would do

cd $GOROOT/src/github.com/joe/project/ && bazel build ...

[thefallentree]

I couldn't beleive how is that acceptable.

Supporting go get is one matter, supporting go build is entirely another (which I don't like either, that's why things like http://getgb.io/ exist. I don't quite believe that's the correct direction we are heading.

Say I have a workspace today, when someone decided to add a go project into it, suddenly I need to modify my "workspace" directory in my home machine to build it....

[hanwen]

You only have to modify your workspace directory if you want to use the open source 'go' tool.

You also don't need to modify it per se. You could also do

mkdir $GOROOT/src/github.com/joe/ ln -s my-bazel-workspace $GOROOT/src/github.com/joe/project

to get the same effect.

[thefallentree]

ok, now I understand, it's not that bad. still, trying to make bazel workspace work with go build is totally backward. This should be totally a per-user choice, it is almost as ridiculous as asking bazel java rule to work with maven. (Except of course, apachers don't have authority over us googlers , so it would not happen anyway).

I couldn't believe this is the reason we don't yet have a functioning golang rule. If people want to use go build tool for some unknown reason, they should setup symlinks under their GOROOT directory directly themselves. not forcing bazel people to do it.

[hanwen]

landed in https://github.com/bazelbuild/bazel/commit/47182329400e359a7a6ad5857cf1854a4f9ec720

there are some bits missing, which should start appearing over the coming weeks.

[jiridanek]

@thefallentree The go tool is considered to be an important part of the Go development experience and must not be broken. It provides very important consistency across different go codebases, the same way that the formatter or testing package do. See https://youtu.be/XvZOdpd_9tc?t=7m36s ("My next example is testing...") to get a better feel for that attitude. Then "shared conventions" part at 11:30.

What Bazel can do for go is to 1) build mixed language projects 2) provide dependency version management. It should build on go build and deliver more, not replace it and deliver differently.

edit: dependency version management

[thefallentree]

@jirkadanek That's a interesting take. I have the opposite view , go should just provide a general tool to build/link go files (go build can do that). It should now not dictate how I structure my workspace for "other stuff", or even go files. right now the bad part of "go build" do that, but I can choose not to use it.

However force bazel to workaround "go build" limitations, is hilarious. The main selling point for bazel is that we structure codebase as small reusable units that can be easily combined together. How is this related to "go build" (notice: not go) , is beyond me.

If you want to make go get/build work under bazel, try making changes in them directly, not bazel!

[hanwen]

@thefallentree - Bazel is a Google project, and Go is a google project too, and that means that we do as the Go team asks us to do. They explicitly asked us to design rules that interoperate with "go build". If you think that is the wrong stance to take, you can try to make your point with the Go team.

Also, these rules are not built in to Bazel, so if you want to make rules that work differently and break all conventions, you are free to do so in any way you please.

[thefallentree]

Well, You have me there. I can not talk about bazel stuff in bazel forum, now I need to go talk with go team, because they are the ones actually in charge. No thanks, I will probably just go build my own rules. Or use this one, who knows. because no one really cares what I do.

[hanwen]

if noone really cares, I guess we can close this discussion then.

I'm closing this issue, also because the rules were shipped. We can have separate issues for separate features.

[mzhaom]

I don't agree that go and bazel should be treated equally even if they are both Google projects. And in this particular case, it's not the whole Go project but just the "go build" tool, which is designed to only build go code in certain way.

I understand the motivation that "go build" tool wants to dedicate how the source tree should be organized. Sometimes it actually makes life easier. "go build" is designed to just build go code and it's a very simple to use choice. If certain people are happy with it and want to make sure it works with their code base, that's their choice and they shouldn't need to use bazel at all because "go build" should "just work"(TM). On the other hand, Go is not the only programming language in the world yet. One particular reason I like bazel is it provides a possibility to put code in different languages together and build them in a consistent way. IMO, a design choice should be made such a use case works well even if Go is present in the code base.

Any way, at least the current go rules are implemented as skylark extension so they can be easily customized. Personally I like kythe go rule better and I have the freedom to choose it instead. But as a project, I wish bazel should focus on providing an out-of-box solution for the major users it tries to target, so please do think again for this particular decision. Thank you!

[hanwen]

I don't understand the fuss.

You should be able to get effect of Kythe's Go (or Buck's for that matter) rules by setting go_prefix("").

If you don't want to be interoperable, nobody is forcing you. It was however an explicit request to leave open this possibility.

[perezd]

I think the attitude is a little worrisome. The fact that they are both Google projects seems like not a great reason to just do whatever Google wants. The criteria should be is this the right solution for people who choose bazel as a build tool explicitly.

On Fri, Oct 16, 2015, 9:58 AM Han-Wen Nienhuys notifications@github.com wrote:

I don't understand the fuss.

You should be able to get effect of Kythe's Go (or Buck's for that matter) rules by setting go_prefix("").

If you don't want to be interoperable, nobody is forcing you. It was however an explicit request to leave open this possibility.

— Reply to this email directly or view it on GitHub https://github.com/bazelbuild/bazel/issues/79#issuecomment-148770948.

[hanwen]

The design for these rules was posted on bazel-dev@ on August 18th. See https://groups.google.com/forum/#!searchin/bazel-dev/go$20support/bazel-dev/xNljbptOfKU/wuYDl_voAwAJ

There have been no comments whatsoever about the design by people that are chiming in now. If you care about the what Go support should look like, perhaps you could have engaged at an earlier point in time?

From this thread, I miss specific suggestions about what the problem is, and I would welcome concrete, actionable feedback.

Thanks,

[davidzchen]

I am wondering whether the GitHub Issues might be more discoverable/visible than the bazel-dev@ mailing list. I am not sure about others but I usually visit my GitHub notifications first before the mailing list.

Perhaps in the future, it might be a good idea to open an issue on GitHub for getting feedback, similar to how Neovim has issues tagged as [RFC] on their issue tracker. Another thing we might consider is having "umbrella" issues; GitHub Issues has a way to create a list of sub-tasks for a given issue.

As for the Go convention, I think it is a good idea to be able to be compatible with the go tool because there would be users who might still want to enable using the go tool for certain Go packages in their projects, such as allowing other non-Bazel Go projects to depend on them.

I definitely agree that we need ensure that Bazel has an excellent out-of-box experience, and we avoid fragmentation of rule implementations whenever possible. Perhaps we can improve the way we convey and document features in order to improve clarity and avoid confusion.

As @hanwen mentioned, the design doc for the Go rules is here: https://docs.google.com/document/d/1_fNIFC-2gLETPBXWipNDaZEi1NCKDJRBqGN7lF0Eo7k/edit#heading=h.s2m2zdq6nrbz

Now that the initial implementation is available in Bazel to try, let's work on improving the Go rules and open separate issues for individual improvements. Perhaps we can also see if we can merge some of the features in Kythe's implementation with the official Bazel one? I think it would be best to not have separate implementations of the same feature among different Google projects. :)

[ghost]

Suggest create github issues with RFC:support golang programming langugage build

Then we should see the document well. Or create sperate repository like golang proposal

https://github.com/golang/proposal

[ulfjack]

I agree with David.

I think the Go team agrees that we ultimately want to serve our users, not the other way 'round. My impression is that they do listen to concerns and are happy to answer questions.

I see interop with the existing go tooling as an important feature to make it easy to migrate to Bazel, import third party Go code into a project that uses Bazel or even to migrate away from Bazel. While the source layout encouraged by the Go team is a bit unusual, I don't see any strong technical reasons that it's bad in some way. Let's start with making the rules interop with the existing tooling, and focus on specific technical issues when they come up.

So, if you have specific concerns, please do bring them up.

[hanwen]

@netroby - Go team defined a RFC model, because there were a lot of proposals being brought up over and over again. I don't think we have the same problem.

[pspeter3]

Reading through the Go rules, it seems like you would need to

• Put your workspace at the correct location in the $GOPATH
• Use their vendoring solution as opposed to Bazel's built in ways for downloading 3rd party dependencies.

Are these assumptions correct?

[hanwen]

The $GOPATH location is optional.

I don't think our downloading approach will work well, since you'd have to overlay a BUILD file in each directory of the tarball/git repo.

[pspeter3]

Would these rules also work for the Go AppEngine SDK?

[damienmg]

These rules have landed at tools/build_defs/go and do not support Go AppEngine SDK. We have support for Java AppEngine and we will welcome any contribution for the Go or Python support but I am afraid we have not enough resource to work on that right now.

[hanwen]

AFAICR, to package an application for AppEngine SDK, you have to stick the sources into a correctly layed out tarball. You should be able to write a rule for that, using the provider for "sources".

[aschran]

The doc posted on August 24 discusses your plans to release a Bazel version of the "glaze" tool for generating BUILD files. The Go language support issue is closed but I think that still remains to be done. Any roadmap updates you can offer on the glaze tool (or the other items in that doc that are not yet complete)? I couldn't find a followup sub-issue tracking those.

Link for reference: https://docs.google.com/document/d/1_fNIFC-2gLETPBXWipNDaZEi1NCKDJRBqGN7lF0Eo7k/view

[davidzchen]

@aschran - Can you open a separate issue for glaze? Thanks!

[aschran]

Done, thanks! https://github.com/bazelbuild/bazel/issues/758

[liues1992]

Please don't support golang. go build is good enough.