elixir-nx / xla

Pre-compiled XLA extension
Apache License 2.0
83 stars 21 forks source link

Cannot build on FreeBSD 14 x86_64 #78

Closed leonardorame closed 3 months ago

leonardorame commented 4 months ago

Hi, trying to test this on LiveBook running on FreeBSD 14 x86_64, but I get this error:

Generated xla app
** (CaseClauseError) no case clause matching: ["x86_64", "unknown", "freebsd14.0"]
    (xla 0.5.1) lib/xla.ex:143: XLA.target_triplet/0
    (xla 0.5.1) lib/xla.ex:121: XLA.target/0
    (xla 0.5.1) lib/xla.ex:117: XLA.archive_filename_with_target/0
    (xla 0.5.1) lib/xla.ex:164: XLA.archive_path_for_matching_download/0
    (xla 0.5.1) lib/xla.ex:32: XLA.archive_path!/0
    mix.exs:112: EXLA.MixProject.extract_xla/1
    (mix 1.16.1) lib/mix/task.ex:557: Mix.Task.run_alias/6
    (mix 1.16.1) lib/mix/tasks/compile.all.ex:124: Mix.Tasks.Compile.All.run_compiler/2

Can you help me with this?

jonatanklosko commented 3 months ago

We are not considering this system architecture string, I improved it on main. You can workaround the error by setting XLA_TARGET_PLATFORM explicitly. That said, as far as I understand FreeBSD has it's own libc implementation, so we don't have a precompiled binary for that. So to use XLA on that platform you need to build it locally with XLA_BUILD=true (assuming that XLA supports FreeBSD, but since it's UNIX, I should probably be fine).

leonardorame commented 3 months ago

Hi @jonatanklosko, thanks for your answer. Could you explain what should I set in XLA_TARGET_PLATFORM?.

leonardorame commented 3 months ago

Hi again @jonatanklosko, I've set this env var:

XLA_TARGET_PLATFORM=X86_64-FreeBSD XLA_BUILD=true

Then cloned the repository and mix deps.get, then mix compile, but I got this error:

** (Mix) Could not compile with "gmake" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".
jonatanklosko commented 3 months ago

Do you have gmake installed?

leonardorame commented 3 months ago

Yes I have gmake, make, gcc.

jonatanklosko commented 3 months ago

You can try running with MAKE=make, MAKE=gmake, I'm not sure what else it can be. If neither works, you can try invoking make explicitly like ROOT_DIR="$(pwd)" BUILD_ARCHIVE="$(pwd)/tmp/build/xla_extension.tar.gz" make to see if there are any meaningful errors.

leonardorame commented 3 months ago

Running this on the cloned repo:

ROOT_DIR="$(pwd)" BUILD_ARCHIVE="$(pwd)/tmp/build/xla_extension.tar.gz" make

First asked for bazel program, so pkg install bazel, then tried again.

Then it needed numpy, so I installed py39-pip, then pip-3.9 install numpy, and tried again.

Then it looks like the source needs g++ 14 minimum, now trying to install it.

leonardorame commented 3 months ago
CXXOPTS="-std=c++14" CC=gcc14 CPP=gcc14 ROOT_DIR="$(pwd)" BUILD_ARCHIVE="~/xla_extension.tar.gz" make
ERROR: /usr/home/informemedico/.cache/bazel/_bazel_informemedico/1fd8e3102f291348a56716ebce84006d/external/llvm-project/llvm/BUILD.bazel:360:11: Compiling llvm/lib/TableGen/StringMatcher.cpp [for tool] failed: (Exit 1): clang failed: error executing command (from target @llvm-project//llvm:TableGen) /usr/bin/clang -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF ... (remaining 65 arguments skipped)
In file included from external/llvm-project/llvm/lib/TableGen/StringMatcher.cpp:13:
In file included from external/llvm-project/llvm/include/llvm/TableGen/StringMatcher.h:16:
In file included from external/llvm-project/llvm/include/llvm/ADT/StringRef.h:12:
external/llvm-project/llvm/include/llvm/ADT/DenseMapInfo.h:264:26: error: no template named 'tuple_element_t' in namespace 'std'; did you mean 'tuple_element'?
    using EltType = std::tuple_element_t<I, Tuple>;
                    ~~~~~^
jonatanklosko commented 3 months ago

That error looks still related to the compiler version. On CI we actually build with gcc 9, so worth trying that.

leonardorame commented 3 months ago

Hi @jonatanklosko yes, just pasted that here to document my progress on buiding it. Also asked some FreeBSD folks on the #freebsd channel at irc.libera.chat to help with thiis.

leonardorame commented 3 months ago

Hi @jonatanklosko as you can see here: https://forums.freebsd.org/threads/trying-to-build-a-linux-c-program-on-freebsd-14-with-gcc9.92731/ I'm asking for help in the FreeBSD forums.

One conclusion I came up is that the Bazel is using /usr/bin/clang, do you think it can be the culpit of my problem. Do you know how to change it to, for example, /usr/local/bin/gcc9?.

leonardorame commented 3 months ago

Just figured out the issue was in OpenXLA, now asking there: https://github.com/openxla/xla/issues/10630

leonardorame commented 2 months ago

I think I made it work in a Linux Jail!

image

Now I need a small example to test.