Support Windows builds through clang

[therc]

Right now, the tested configurations for Linux, Mac and Windows use gcc, clang and MSVC, as seen at https://www.tensorflow.org/install/install_sources#tested_source_configurations

Linux can be made to compile with clang, too, if you use the right magic trick (--config=cuda_clang). All that's left is Windows, which is probably the trickiest of the three.

Bonus points for allowing to cross-build for Windows under Linux. The main problem there might be with CUDA and getting its SDK installed on a Linux system (just a wild guess).

(Forked from an existing discussion under https://github.com/tensorflow/tensorflow/issues/12052)

[mrry]

This sounds potentially useful for development, but it's unlikely anyone on the team will take this on soon, because we already have a supported configuration that works well enough. Marking it as contributions welcome, in the hope that somebody wants to take it on.

[AtlantaPepsi]

sounds greatly interesting, I'll try to investigate into this.

[AtlantaPepsi]

I see that @gunan has written a windows clang toolchain under third_party/toolchains/preconfig/win/bazel_211, I'll probably assemble a new configuration with that and appropriate platform with new constraints. Right now I'm stuck on getting bazel working on my local machine, similar to #31608, where undname.exe couldn't be found under VC directory

[gunan]

That toolchain was constructed for MSVC 2019. AFAICR, all clang references to it are placeholders @artem-b may be aware if we have windows clang toolchains checked in.

[Artem-B]

I don't think you need one. I've been using the default one provided by bazel. I've basically followed these instructions: https://docs.bazel.build/versions/master/windows.html#build-c-with-clang

[AtlantaPepsi]

@Artem-B Just that will do? I had seen those instructions, but was wondering if that's too simple for a "tricky" problem, then I realized this issue was opened back in 2017, perhaps Bazel didn't support this option back then.

But the platform target still need to be manually added by users afterwards right? I don't see it written anywhere in the source code. Do you think it would be still useful for me to add it somewhere and make a new config for clang on windows by incorporating those options and modifying the config.py ?

Another thing, MSVC seems to be the only option for CUDA compiler, I'm not sure how relatable is it to this issue or if it's already an open issue somewhere, but it might also be useful to include clang in CUDA configuration for windows?

[Artem-B]

BUILD/WORKSPACE files will need to have these modifications, though I think they will eventually make it into TF. Adding it all to the config.py may be a bit premature -- people will run with it and will start complaining when things don't quite work, and builds on Windows do have a lot of sharp corners.

As for CUDA compilation, clang should be able to do it in principle, but TF build needs a bit of work to put things together. Basically the issue is that windows build with clang uses clang-cl which is a drop-in replacement for MSVC's cl and uses different command-line syntax. As far as bazel is concerned, CUDA compilation with clang is another C++ compilation with a few extra options. The problem is that TF currently only constructs these extra options to be used by regular clang and will need to prefix all of them with /clang (I think) if they were to be used with clang-cl on windows.

MSVC seems to be the only option for CUDA compiler,

Do you mean NVCC? MSVC can't compile CUDA. That said, if you build with clang, your only choice is to whether enable CUDA or not. You can't (well, shouldn't, really) use NVCC in this case. I vaguely recall having to tweak config.py to force-enable CUDA for the build w/ clang.

[mohantym]

Hi @therc! We are checking whether you are still looking help in this issue or not. You can say yes to clang support while running configure.py in after checking out r2.8/2.9 branch.

Attached relevant thread from source. Thank you!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[mihaimaruseac]

CC @angerson @learning-to-play as there is some testing to use clang for building.

[learning-to-play]

@bani-intelaipg

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. It will be closed if no further activity occurs. Thank you.

[japm48]

According to the release page, clang is now used for the official TF builds of version 2.16.1. The problem is there is no documentation (that I could find) on how to replicate that.

[learning-to-play]

@mraunak

[mraunak]

Hi @japm48, Thank you for your patience. The documentation will be available soon. Please follow the steps below to build TensorFlow with CLANG on Windows

Install LLVM Go to the LLVM downloads{:.external}, Download and install Windows-compatible LLVM in C:/Program Files/LLVM e.g., LLVM-17.0.6-win64.exe (recommended)

Note: To build with Clang on Windows, it is required to install both LLVM and Visual C++ Build tools as although Windows uses clang-cl.exe as the compiler, Visual C++ Build tools are needed to link to Visual C++ libraries

Environmental set up

set BAZEL_SH=C:/msys64/usr/bin/bash.exe 
set BAZEL_VS=C:/Program Files/Microsoft Visual Studio/2022/BuildTools 
set BAZEL_VC=C:/Program Files/Microsoft Visual Studio/2022/BuildTools/VC 
set Bazel_LLVM=C:/Program Files/LLVM (explicitly tell Bazel where LLVM is installed by BAZEL_LLVM, needed while using CLANG)
set PATH=C:/Program Files/LLVM/bin;%PATH% (Optional, needed while using CLANG as Compiler)

Build with CLANG
Use --config=win_clang to build TenorFlow with the CLANG Compiler:

bazel build --config=win_clang --repo_env=TF_PYTHON_VERSION=3.10 //tensorflow/tools/pip_package:wheel --repo_env=WHEEL_NAME=tensorflow_cpu

[japm48]

Thanks @mraunak !

I got this error: no such target '//tensorflow/tools/pip_package:wheel'.

So, I tried with //tensorflow/tools/pip_package:build_pip_package.

Also, the build_pip_package executable requires rsync on msys2 to work.

Apart from that, build_pip_package complained about invalid command 'bdist_wheel', even if wheel package was installed. After some research I found that it uses the python executable found in bazel-tensorflow/external, so installing wheel in that did the trick.

---

For the record, the complete setup is as follows (all commands executed in an "as admin" cmd.exe prompt):

• VS2022 Tools: current latest version.
• MSYS2 installed at C:\msys64 updated and with required packages installed:
  • pacman -Syu (requires a console restart)
  • pacman -S git patch unzip rsync
• Python 3.10 installed at C:\Python310
  • C:\Python310\python.exe -m pip install -U pip (to update pip itself)
  • C:\Python310\python.exe -m pip install -U six numpy wheel packaging
  • C:\Python310\python.exe -m pip install -U keras_preprocessing --no-deps
• bazelisk installed and reachable in PATH (e.g. using choco install bazelisk)
• LLVM version 17.0.6, as recommended.
• Tensorflow 2.16.1 downloaded from the releases page.
  • Folder containing the sources must be named tensorflow, and not tensorflow-2.16.1 (otherwise build_pip_package fails).

---

Now to build it (again, using cmd.exe as admin):

set BAZEL_SH=C:/msys64/usr/bin/bash.exe
set BAZEL_VS=C:/Program Files/Microsoft Visual Studio/2022/BuildTools
set BAZEL_VC=C:/Program Files/Microsoft Visual Studio/2022/BuildTools/VC 
set Bazel_LLVM=C:/Program Files/LLVM
set PATH=C:/Python310/;C:/Python310/Scripts;C:/Program Files/LLVM/bin;%PATH%

bazel build --config=win_clang --repo_env=TF_PYTHON_VERSION=3.10 //tensorflow/tools/pip_package:build_pip_package --repo_env=WHEEL_NAME=tensorflow_cpu

And finally:

.\bazel-tensorflow\external\python_x86_64-pc-windows-msvc\python.exe -m pip install wheel
.\bazel-bin\tensorflow\tools\pip_package\build_pip_package ./tensorflow_pkg

---

I would say this should be documented in the build instructions page before closing this issue (you can copy any of the above if necessary).

[mraunak]

Hi @japm48, Thank you very much! I will incorporate the changes suggested by you. We recently changed the target to build the TensorFlow(https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/pip_package/BUILD#L265) Request you to clone to the latest TensorFlow Repo and try to build the target //tensorflow/tools/pip_package:wheel

[japm48]

@mraunak I will try again with the latest dev version when I have the time and resources, and will report back. The issue with the missing wheel dependency is perhaps fixed in that version.

Just a quick question: given the presence of the new "hermetic Python", is it no longer necessary to manually install Python itself on the build system?

[mraunak]

yes, the Hermetic Python will disregard all Python package installations on the machine and you will get the same behavior if your machine is plain and doesn't have Python (or its packages) installed.

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. It will be closed if no further activity occurs. Thank you.

[japm48]

Forgot this existed, sorry. I'll have a go in the next few days to confirm if this is enough.