Tensorflow + Xilinx SYCL

triSYCL / sycl

SYCL for Vitis: Experimental fusion of triSYCL with Intel SYCL oneAPI DPC++ up-streaming effort into Clang/LLVM

Other

107 stars 19 forks source link

Tensorflow + Xilinx SYCL #74

Closed powderluv closed 4 years ago

powderluv commented 4 years ago

Just poking around OpenCL/SYCL support in TF. Im wondering if I can use this fork of sycl as a backend to TF (and use 2019.2 v++ in the intermediate stage) and target the Alveo U2xx devices ? Thx

keryell commented 4 years ago

It is something we tried a few years ago but we gave up. Just a few problems:

TensorFlow SYCL is a generalization of TensorFlow CUDA and is optimized for GPU architecture and would have very poor performance on FPGA;
TensorFlow SYCL relies on Eigen meta-programming magic like TensorFlow CUDA. The side effect is that all the possible kernels are instantiated, which is not a big problem for GPU compilers, but a huge one on FPGA, where compiling just one kernel is already time consuming;
TensorFlow SYCL is a huge application, quite too challenging for our very partial SYCL implementation.

So you will have a better experience with official Xilinx ML environments dealing with TensorFlow.

Otherwise, generating some SYCL code from MLIR or other ML compiler frameworks is also quite appealing, since you can direct your code generator to generate various SYCL code according to the architecture family.

powderluv commented 4 years ago

Thank you for the valuable insights.

I was hoping Xilinx SYCL could/would be a first class citizen in TF via TF SYCL, especially for training acceleration. I hope the three issues you listed can be addressed. Most of the Xilinx ML environments assumes training is done elsewhere (pls lmk if this is not true). We have a few training acceleration blocks that can run on an Alveo that I am trying to plumb into the TF framework properly (instead of the custom hack we have in TF right now)

MLIR is a good option for the inference pipeline and we have plugins to farm out to different accelerators but training support is lacking. Eventually looks like MLIR will replace SavedModel as the on disk format too and also support training but that seems like a ways away.

keryell commented 4 years ago

@powderluv since I have seen recently a presentation from https://nod.ai. I guess that you are connected with the right persons at Xilinx for some alternative. Ping me privately if you do not get support.

powderluv commented 4 years ago

Yes yes thank you. I gave the presentation and saw you on the meeting roster but couldn't say hi then due the size of the call. We are connected to the right teams. Will keep you posted on progress directly.

Closing this issue.