Bazel

[sayrer]

I started working on getting this project to compile with Bazel and BuildBuddy. This effort will result in reproducible builds and cached build results (meaning that something already done on a branch won't require a recompile).

I'll need to use cargo-raze to pull in the Cargo deps. But, this way, the project will have a build that pulls in a fixed Rust and C/C++ toolchain. (Bazel downloads it all, and doesn't depend on the local machine)

https://github.com/grafica/crustls/tree/buildbuddy

[sayrer]

First try: https://app.buildbuddy.io/invocation/ee9c8180-46fd-4834-9636-e66cc3a48df5

Failed because none of the Cargo libs are downloaded for Bazel (yet).

[sayrer]

BuildBuddy is buggy in building the cargo-raze tool itself, but things are working fine so far on my local Ubuntu x64 system. I have Bazel automatically producing all of the dependencies from Cargo, so the next step should be to get the Rust files compiling.

(I've talked with the BuildBuddy folks before, so I sent them feedback on this)

[sayrer]

This vendors all of the Cargo stuff for Bazel, and produces a working BUILD.bazel file at the top level for them.

https://github.com/grafica/crustls/tree/de3a56683dc436bfc118395fbc86232489aa61a9

I'll see if I can get the crustls Rust files compiling now.

[sayrer]

Almost works now, except the Bazel build of Ring doesn't put some data files in the Bazel sandbox:

INFO: From Compiling Rust rlib ring v0.16.20 (86 files):
error: couldn't read cargo/vendor/ring-0.16.20/src/ec/curve25519/ed25519/ed25519_pkcs8_v2_template.der: No such file or directory (os error 2)
   --> cargo/vendor/ring-0.16.20/src/ec/curve25519/ed25519/signing.rs:266:12
    |
266 |     bytes: include_bytes!("ed25519_pkcs8_v2_template.der"),

The file is there, Bazel just hasn't tracked it and included it in its sandbox:

$ ls cargo/vendor/ring-0.16.20/src/ec/curve25519/ed25519/ed25519_pkcs8_v2_template.der
cargo/vendor/ring-0.16.20/src/ec/curve25519/ed25519/ed25519_pkcs8_v2_template.der

This can be fixed by adding some Cargo Raze data directives to the Cargo.toml file.

[sayrer]

Looks like a known roadblock: https://github.com/google/cargo-raze/issues/41#issuecomment-638986696

[sayrer]

bazel build //src:crustls_rust works now. There is still a remaining dependency on cbindgen to generate some code. This will be a Bazel genrule.

https://github.com/grafica/crustls/tree/58b7d7aafe1d204043229ba57c9b576ad38ffa9b

[sayrer]

Just a note on using the "vendor" strategy for dependencies here: Cargo will download them anyway, checking them into the source tree just makes sure the Bazel build uses the tested ones.

[sayrer]

I got this working on my branch. This will provide a reproducible build all the way down to the toolchains, I hope. As a result, it will take quite a while the first time you run it. I hope to get BuildBuddy working here, so that the cloud will have cached artifacts for most of it.

Sample output:

sayrer@sayrer-ml:~/github/grafica$ git clone https://github.com/grafica/crustls
Cloning into 'crustls'...
remote: Enumerating objects: 9028, done.
remote: Counting objects: 100% (9028/9028), done.
remote: Compressing objects: 100% (4100/4100), done.
remote: Total 9735 (delta 4812), reused 9002 (delta 4792), pack-reused 707
Receiving objects: 100% (9735/9735), 15.79 MiB | 15.31 MiB/s, done.
Resolving deltas: 100% (5242/5242), done.
sayrer@sayrer-ml:~/github/grafica$ cd crustls/
sayrer@sayrer-ml:~/github/grafica/crustls$ git checkout buildbuddy
Branch 'buildbuddy' set up to track remote branch 'buildbuddy' from 'origin'.
Switched to a new branch 'buildbuddy'
sayrer@sayrer-ml:~/github/grafica/crustls$ bazel --version
bazel 4.0.0
sayrer@sayrer-ml:~/github/grafica/crustls$ ./update_bazel.sh 
    Updating crates.io index
To use vendored sources, add this to your .cargo/config.toml for this project:

[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "cargo/vendor"
Starting local Bazel server and connecting to it...
INFO: Analyzed target @cargo_raze//:raze (209 packages loaded, 44431 targets configured).
INFO: Found 1 target...
INFO: Deleting stale sandbox base /home/sayrer/.cache/bazel/_bazel_sayrer/8e5e07a0e1f8c164a058bc6d24e62a87/sandbox
Target @cargo_raze//tools:raze up-to-date:
  bazel-bin/external/cargo_raze/tools/raze
INFO: Elapsed time: 7.352s, Critical Path: 0.17s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
sayrer@sayrer-ml:~/github/grafica/crustls$ bazel run //src:main www.google.com /  > google_output.txt
Loading: 
Loading: 0 packages loaded
Analyzing: target //src:main (1 packages loaded, 0 targets configured)
INFO: Analyzed target //src:main (48 packages loaded, 2515 targets configured).
INFO: Found 1 target...
[0 / 6] [Prepa] BazelWorkspaceStatusAction stable-status.txt
Target //src:main up-to-date:
  bazel-bin/src/main
INFO: Elapsed time: 1.397s, Critical Path: 0.07s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/src/main www.google.com /
INFO: Build completed successfully, 1 total action
ClientSession wants us to write_tls.
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to write_tls.
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
reading from socket: EAGAIN or EWOULDBLOCK: Resource temporarily unavailable
ClientSession wants us to read_tls. First we need to pull some bytes from the socket
EOF reading from socket
sayrer@sayrer-ml:~/github/grafica/crustls$ more google_output.txt 
HTTP/1.1 200 OK
Date: Wed, 24 Mar 2021 23:49:40 GMT
Expires: -1
Cache-Control: private, max-age=0
...

[sayrer]

I dropped a PR in #78 as a start.

[sayrer]

@jsha indicated he doesn't really want this right now (that's fine), but I thought I'd point out that the patch is a lot smaller than it looks, because I mostly checked in generated files. The code I wrote by hand for Bazel ended up being about 4-5 files of easy scripting.

Here's the dep tree for this project: https://gist.github.com/sayrer/bd8794e118107980eae65232c1f42b68

And here's an optimized build done in 11 seconds from a fresh checkout, after having done it once before: https://app.buildbuddy.io/invocation/edd233e4-a12e-4475-9279-d59483677db0

Notice that the cache hit rate is super high. This gets you a build with fixed versions of llvm/clang, rustc, cargo, cbindgen, and every crate in the project (like rustls, webpki, and ring), and everything like timestamps meticulously removed so builds stay the same. That's only a little bit faster than what the Threadripper machine in my office can do from scratch with cargo, but that build depends on that machine's config. I also didn't run any tests in Bazel, where it's pretty good at caching things.

As this project matures, I'd expect a reproducible build like this to be a security requirement (Bazel is what I know, but there are other ways to do it).

[cpu]

jsha indicated he doesn't really want this right now (that's fine)

I think that remains our position.

I'm going to close this issue since it's not an active goal but if there are downstream users that would benefit from bazel and they're willing to help with the added maintenance burden, please comment and we can re-open/re-visit.