Support for XLA based devices

flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

https://flexflow.readthedocs.io

Apache License 2.0

1.59k stars 218 forks source link

Closed mmcclean-aws closed 3 months ago

mmcclean-aws commented 7 months ago

Does FlexFlow have the capability to support XLA based devices (e.g. TPU, Trainium) or is it tied to Cuda ?

lockshaw commented 3 months ago

Currently tied to CUDA unfortunately