Includes needed for go asm missing from sandbox

[sqwishy]

I'm unable to build a project that uses https://github.com/o1egl/paseto because bazel fails to build one of that project's dependencies. Below is the build output and a diagnosis concerning the sandbox that bazel creates. Later there is a WORKSPACE and source file that might make a minimal reproducible example.

The build output (right after bazel clean):

$ bazel build ... --sandbox_debug --verbose_failures
INFO: Analysed 2 targets (23 packages loaded).
INFO: Found 2 targets...
ERROR: /home/itrick/.cache/bazel/_bazel_itrick/3da33cce868be8ce123b5a603f89d38e/external/com_github_aead_chacha20/chacha/BUILD.bazel:3:1: GoAsm external/com_github_aead_chacha20/chacha/linux_amd64_stripped/go_default_library~/chacha_amd64.o failed (Exit 1): process-wrapper failed: error executing command 
  (cd /home/itrick/.cache/bazel/_bazel_itrick/3da33cce868be8ce123b5a603f89d38e/execroot/__main__ && \
  exec env - \
    CGO_ENABLED=1 \
    GOARCH=amd64 \
    GOOS=linux \
    GOROOT='bazel-out/k8-fastbuild/bin/external/io_bazel_rules_go/linux_amd64_stripped/stdlib~' \
    GOROOT_FINAL=GOROOT \
    PATH=/usr/bin \
    TMPDIR=/tmp \
  /home/itrick/.cache/bazel/_bazel_itrick/3da33cce868be8ce123b5a603f89d38e/execroot/__main__/_bin/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/host/bin/external/io_bazel_rules_go/go/tools/builders/linux_amd64_stripped/asm -go external/go_sdk/bin/go external/com_github_aead_chacha20/chacha/chacha_amd64.s -- -I 'bazel-out/k8-fastbuild/bin/external/io_bazel_rules_go/linux_amd64_stripped/stdlib~/pkg/include' -trimpath . -o 'bazel-out/k8-fastbuild/bin/external/com_github_aead_chacha20/chacha/linux_amd64_stripped/go_default_library~/chacha_amd64.o')
external/com_github_aead_chacha20/chacha/chacha_amd64.s:8: #include: open /home/itrick/.cache/bazel/_bazel_itrick/3da33cce868be8ce123b5a603f89d38e/bazel-sandbox/8310765647797671785/execroot/__main__/bazel-out/k8-fastbuild/bin/external/io_bazel_rules_go/linux_amd64_stripped/stdlib~/pkg/include/const.s: no such file or directory
GoAsm: error running subcommand: exit status 1
INFO: Elapsed time: 10.107s, Critical Path: 7.61s
FAILED: Build did NOT complete successfully

The chacha20 project has a go asm file, chacha_amd64.s that includes a sibling file, that include fails, but only when run in the bazel sandbox.

If I run the command that bazel build shows me, the compilation is successful -- because it says to cd into a directory that is not the sandbox. But bazel runs the compilation inside of the sandbox directory. The build is successful when run under BAZEL_CACHE/3da33cce868be8ce123b5a603f89d38e/execroot/__main__ but not BAZEL_CACHE/3da33cce868be8ce123b5a603f89d38e/bazel-sandbox/8310765647797671785/execroot/__main__

When the build does work, the include is found at external/com_github_aead_chacha20/chacha/const.s.

From outside the sandbox, at BAZEL_CACHE/3da33cce868be8ce123b5a603f89d38e/execroot/__main__

~/.c/b/_/3/e/__main__> ls external/com_github_aead_chacha20/chacha/
BUILD.bazel    chacha_386.s     chacha_amd64.s      chacha_generic.go  chacha_ref.go   const.s
chacha_386.go  chacha_amd64.go  chachaAVX2_amd64.s  chacha.go          chacha_test.go  macro.s

From inside the sandbox at BAZEL_CACHE/3da33cce868be8ce123b5a603f89d38e/bazel-sandbox/8310765647797671785/execroot/__main__

~/.c/b/_/3/b/8/e/__main__> ls external/com_github_aead_chacha20/chacha/
chacha_amd64.s@

After adding symlinks in the sandbox for the two includes, const.s and macro.s, the build is successful.

The following is the WORKSPACE and a go source file that imports a package that depends on the one that can't be built.

~/go/src/meme/WORKSPACE:

http_archive(
    name = "io_bazel_rules_go",
    url = "https://github.com/bazelbuild/rules_go/releases/download/0.12.0/rules_go-0.12.0.tar.gz",
    sha256 = "c1f52b8789218bb1542ed362c4f7de7052abcf254d865d96fb7ba6d44bc15ee3",
)

http_archive(
    name = "bazel_gazelle",
    url = "https://github.com/bazelbuild/bazel-gazelle/releases/download/0.12.0/bazel-gazelle-0.12.0.tar.gz",
    sha256 = "ddedc7aaeb61f2654d7d7d4fd7940052ea992ccdb031b8f9797ed143ac7e8d43",
)

load(
    "@io_bazel_rules_go//go:def.bzl",
    "go_rules_dependencies",
    "go_register_toolchains",
)

go_rules_dependencies()

go_register_toolchains()

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")

gazelle_dependencies()

go_repository(
    name="org_golang_x_crypto",
    importpath="golang.org/x/crypto",
    commit="master",
)

go_repository(
    name="com_github_aead_poly1305",
    importpath="github.com/aead/poly1305",
    commit="969857f48f7ae439b6d2449ed1dcd9aaabc49c67",
)

go_repository(
    name="com_github_aead_chacha20",
    importpath="github.com/aead/chacha20",
    commit="e2538746bfea853aaa589feb8ec46bd46ee78f86",
)

go_repository(
    name="com_github_aead_chacha20poly1305",
    importpath="github.com/aead/chacha20poly1305",
    commit="233f39982aeb502bfca81558666747bec91cf00c",
)

go_repository(
    name="com_github_o1egl_paseto",
    importpath="github.com/o1egl/paseto",
    tag="v0.2.0",
)

go_repository(
    name="org_golang_x_crypto",
    importpath="golang.org/x/crypto",
    commit="master",
)

go_repository(
    name="org_golang_x_sys",
    importpath="golang.org/x/sys",
    commit="c11f84a56e43e20a78cee75a7c034031ecf57d1f",
)

~/go/src/meme/main.go:

package main

import (
    "log"

    "github.com/o1egl/paseto"
)

func main() {
    jsonToken := paseto.JSONToken{Subject: "potato"}
    log.Println(jsonToken)
}

[sqwishy]

It seems to me that the included files, const.s, macro.s, are treated as asm sources (rather than headers?) due to their file extension. So then I think go.asm/emit_asm isn't invoked with all the files necessary for bazel to compile the target.

This diff produces behaviour that allows the build to succeed; although I'm sure it's being gratuitous.

diff --git a/go/private/actions/archive.bzl b/go/private/actions/archive.bzl
index 6721c93..c9306ea 100644
--- a/go/private/actions/archive.bzl
+++ b/go/private/actions/archive.bzl
@@ -45,7 +45,8 @@ def emit_archive(go, source=None):

   extra_objects = []
   for src in split.asm:
-    extra_objects.append(go.asm(go, source=src, hdrs=split.headers))
+    headers = split.headers + split.asm
+    extra_objects.append(go.asm(go, source=src, hdrs=headers))

   direct = [get_archive(dep) for dep in source.deps]
   runfiles = source.runfiles

[sqwishy]

Perhaps this is an issue with the generated BUILD file? As I understand, that is produced by gazelle, which is a separate program?

[jayconrod]

Looked into this a bit. I think a fix will have two parts:

• For each GoAsm action, all assembly and header files in that package should be inputs.
• We should pass include directories (-I flags) for all assembly files.

For the record, I think including .s files is a bad idea. rules_go should support this because go build supports it, but I'm pretty sure the resulting binaries will end up with multiple copies of symbols defined in the included files.

[jayconrod]

The diff you posted earlier actually fixes this (no extra munging include directories; that's already done for headers).

1521 is that plus some documentation and a test.