Build/Test is failing on Mac with github.com/coreos/etcd dependency

[kalbasit]

Hello,

My project has a dependency on github.com/coreos/etcd and it works fine on Linux, but it's not working on Mac. I've created a reproducible test at https://github.com/kalbasit/bazel-reproducible/tree/master/coreos_etcd_client

I'm suspecting that it's related to #456, but I've added extra_args = ["-build_file_name", "BUILD.bazel"], to the gazelle rule but it did not help.

Linux:

$ bazel test //...
WARNING: ignoring _JAVA_OPTIONS in environment.
INFO: Analysed 2 targets (26 packages loaded).
INFO: Found 1 target and 1 test target...
INFO: Elapsed time: 1.720s, Critical Path: 0.05s
INFO: 1 process, processwrapper-sandbox.
INFO: Build completed successfully, 2 total actions
//helper:go_default_test                                                 PASSED in 0.0s

Executed 1 out of 1 test: 1 test passes.

Mac:

$ bazel test //...
ERROR: /Users/kalbasit/code/src/github.com/kalbasit/bazel-reproducible/coreos_etcd_client/helper/BUILD.bazel:3:1: no such package '@com_github_coreos_etcd//client': BUILD file not found on package path and referenced by '//helper:go_default_library'
ERROR: Analysis of target '//helper:go_default_library' failed; build aborted: no such package '@com_github_coreos_etcd//client': BUILD file not found on package path
INFO: Elapsed time: 0.246s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (2 packages loaded)
ERROR: Couldn't start the build. Unable to run tests

[kalbasit]

I found another ticket reporting a similar issue #234, @jayconrod you marked #234 as resolved but I'm not sure that it is.

[jayconrod]

Try this instead:

go_repository(
    name = "com_github_coreos_etcd",
    commit = "70c8726202dd91e482fb4029fd14af1d4ed1d5af",
    importpath = "github.com/coreos/etcd",
    build_file_generation = "on",
)

By default, go_repository checks whether there are files named BUILD or BUILD.bazel. If no such file exists, it runs Gazelle. Since this repository has a file named build, you need to force build file generation to be on.

go_repository just passes extra_args to Gazelle directly (if it runs), so that doesn't fix this.

[kalbasit]

@jayconrod is there a way to ask bazel run //:gazelle -- update-repos -from_file=Gopkg.lock to do that?

I have a script that removes all added go_repository from the WORKSPACE so if I add the build_file_generation manually, the script will remove it on the next run.

[jayconrod]

Gazelle only sets a few attributes in go_repository. There's no way to set build_file_generation using Gazelle on the command-line, but it will leave manual edits in place when it updates rules.

Note that gazelle update-repos -from_file=Gopkg.lock will update existing rules if they're present, rather than adding new, redundant rules. So you may not need to remove existing go_repository rules before running gazelle update-repos, unless there's another reason.

[kalbasit]

@jayconrod here's the relevant extract of my script:

#!/bin/bash
set -e

function go_vendor() {
        if [[ "${1}" = "-update" ]]; then
                dep ensure -update
        else
                dep ensure
        fi

        # this package contains a broken symlink and it seems to be killing Bazel
        rm -rf vendor/github.com/coreos/etcd/cmd

        # clear the WORKSPACE of all generated go_repository in preparation of
        # running gazelle update-repos
        sed '/^# gazelle:autogenerated$/q' WORKSPACE > /tmp/WORKSPACE
        mv /tmp/WORKSPACE WORKSPACE

        # update the WORKSPACE
        bazel run --noshow_progress //:gazelle -- update-repos -from_file=Gopkg.lock
}

go_vendor

relevant WORKSPACE section:

####
# NOTE: do not remove or change the line below, `platform vendor` uses this to
# clean the workspace before updating the externals.
#
# gazelle:autogenerated

go_repository(
    name = "com_github_coreos_etcd",
    commit = "70c8726202dd91e482fb4029fd14af1d4ed1d5af",
    importpath = "github.com/coreos/etcd",
    build_file_generation = "on",
)

and of course, when I run it, the flag gets removed

$ git diff WORKSPACE
diff --git a/src/WORKSPACE b/src/WORKSPACE
index 24626637..5cc109ff 100644
--- a/src/WORKSPACE
+++ b/src/WORKSPACE
@@ -166,7 +166,6 @@ go_repository(
     name = "com_github_coreos_etcd",
     commit = "70c8726202dd91e482fb4029fd14af1d4ed1d5af",
     importpath = "github.com/coreos/etcd",
-    build_file_generation = "on",
 )

 go_repository(

The goal of the above script is to keep the WORKSPACE in sync with Gopkg.lock, so whenever we remove a dependency from the lock file, it will also be removed from the WORKSPACE; Of course, breaks when you account for manual changes. Is there a better way to sync them?

[jayconrod]

It seems like the main problem is removing repository rules that are no longer needed. I don't have a better way to do that right now, but I've filed bazelbuild/bazel-gazelle#213 for that. When that's implemented, you could run gazelle update-repos -prune or something similar, and it would remove repository rules that aren't needed anymore.

[kalbasit]

Thanks, @jayconrod. I've removed this functionality for now from the script to prevent confusion among my dev team, and I'll maintain the pruning manually myself.