Target //examples/jax_cpp:main fails to build

[mkovaxx]

Steps to reproduce:

bazel build examples/jax_cpp:main --experimental_repo_remote_exec --check_visibility=false

Output:

((jax) ) mate@raptor:~/prog/jax$ python --version
Python 3.8.10
((jax) ) mate@raptor:~/prog/jax$ pipenv lock -r
#
# These requirements were autogenerated by pipenv
# To regenerate from the project's Pipfile, run:
#
#    pipenv lock --requirements
#

-i https://pypi.org/simple
numpy==1.15.4
((jax) ) mate@raptor:~/prog/jax$ bazel --version
bazel 4.1.0
((jax) ) mate@raptor:~/prog/jax$ bazel build examples/jax_cpp:main --experimental_repo_remote_exec --check_visibility=false
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=104
INFO: Reading rc options for 'build' from /home/mate/prog/jax/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/mate/prog/jax/.bazelrc:
  'build' options: --apple_platform_type=macos --macos_minimum_os=10.9 --announce_rc --define open_source_build=true --spawn_strategy=standalone --enable_platform_specific_config --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true -c opt --config=short_logs
INFO: Found applicable config definition build:short_logs in file /home/mate/prog/jax/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:linux in file /home/mate/prog/jax/.bazelrc: --config=posix --copt=-Wno-stringop-truncation
INFO: Found applicable config definition build:posix in file /home/mate/prog/jax/.bazelrc: --copt=-Wno-sign-compare --cxxopt=-std=c++14 --host_cxxopt=-std=c++14
INFO: Analyzed target //examples/jax_cpp:main (139 packages loaded, 13084 targets configured).
INFO: Found 1 target...
ERROR: /home/mate/.cache/bazel/_bazel_mate/369bb33bd230fc352ed10186a6c50bdf/external/boringssl/BUILD:130:11: Compiling src/third_party/fiat/curve25519.c failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 36 argument(s) skipped)
external/boringssl/src/third_party/fiat/curve25519.c:511:57: error: argument 2 of type 'const uint8_t[32]' {aka 'const unsigned char[32]'} with mismatched bound [-Werror=array-parameter=]
  511 | int x25519_ge_frombytes_vartime(ge_p3 *h, const uint8_t s[32]) {
      |                                           ~~~~~~~~~~~~~~^~~~~
In file included from external/boringssl/src/third_party/fiat/curve25519.c:41:
external/boringssl/src/third_party/fiat/internal.h:117:58: note: previously declared as 'const uint8_t *' {aka 'const unsigned char *'}
  117 | int x25519_ge_frombytes_vartime(ge_p3 *h, const uint8_t *s);
      |                                           ~~~~~~~~~~~~~~~^
external/boringssl/src/third_party/fiat/curve25519.c:831:57: error: argument 2 of type 'const uint8_t *' {aka 'const unsigned char *'} declared as a pointer [-Werror=array-parameter=]
  831 | void x25519_ge_scalarmult_base(ge_p3 *h, const uint8_t *a) {
      |                                          ~~~~~~~~~~~~~~~^
In file included from external/boringssl/src/third_party/fiat/curve25519.c:41:
external/boringssl/src/third_party/fiat/internal.h:125:56: note: previously declared as an array 'const uint8_t[32]' {aka 'const unsigned char[32]'}
  125 | void x25519_ge_scalarmult_base(ge_p3 *h, const uint8_t a[32]);
      |                                          ~~~~~~~~~~~~~~^~~~~
cc1: all warnings being treated as errors
Target //examples/jax_cpp:main failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 161.098s, Critical Path: 29.20s
INFO: 1640 processes: 146 internal, 1494 local.
FAILED: Build did NOT complete successfully

[zhangqiaorjc]

Compiling examples/jax_cpp:main builds a large part of TF/XLA tree. Are you using the same build toolchain required for building TF from source?

If you can build TF from source, you should be able to build jax_cpp:main

https://www.tensorflow.org/install/source

[mkovaxx]

Thanks for taking a look!

I use a combination of pyenv, pipenv, and bazelisk to control the build environment. I can build TF from source in that controlled environment without any issues.

However, in the same environment, //examples/jax_cpp:main fails with the error output above.

Here's a branch with the necessary files added: https://github.com/mkovaxx/jax/commit/e0d49599ab4dd3b6880943f752525755dabd8bc8

And here's the command I use to run the build:

pipenv run bazel build //examples/jax_cpp:main --experimental_repo_remote_exec --check_visibility=false

Do you see anything wrong with this approach?

[zhangqiaorjc]

sorry for the delay; i suspect this is related to the gcc version you are using, and it's more strict that what JAX is using in our vm build

The warnings are coming from boringssl/src/third_party/fiat/curve25519.c

so it's not part of JAX and going to be a bit hard to change.

The easier workaround would be try to use a standard Ubuntu VM to build (https://jax.readthedocs.io/en/latest/developer.html#building-jaxlib-from-source)

@mkovaxx Could you try that?

[hawkinsp]

If that doesn't work for you, can you also (a) try at JAX head and/or (b) provide details of what OS/compiler you are using, ideally enough that we can reproduce this in a VM?

[mkovaxx]

I really appreciate your help on this!

Here's some info about my OS and compiler suite:

mate@raptor:~/prog/jax$ uname -a
Linux raptor 5.15.0-051500-generic #202110312130 SMP Sun Oct 31 21:33:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

mate@raptor:~/prog/jax$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 21.10
Release:    21.10
Codename:   impish

mate@raptor:~/prog/jax$ gcc --version
gcc (Ubuntu 11.2.0-7ubuntu2) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

While compiling Jax-related targets in a VM might work, it would considerably increase the complexity of my build setup so I'd like to find another workaround if possible. For context, I want to use Jax in a C++ project while trying to avoid having it depend on the build environment of Jax (or TensorFlow).

Just as a side note, it comes as a surprise to me that Bazel doesn't pin the compiler suite and version, even though it claims to have "reproducibility" and "hermeticity". (I have some experience with Nix, where "reproducible" and "hermetic" imply that the build is isolated from the surrounding system, including whatever compiler suite happens to be installed system-wide.) I understand that Jax is separate from Bazel, just wanted to explain my thinking here so you could point out any misunderstandings I have.