Open dinghaodhd opened 5 months ago
I don't think it is working with pytorch/XLA now.
I think to make this work;
There will be some exploration done from Google side I believe (cc @allenwang28)
We are exploring TPU compatibility and support in https://github.com/vllm-project/vllm/issues/3620 through PyTorch/XLA.
but after this work, it should be clearer how to adapt PyTorch/XLA in vLLM for different hardware backends CC @WoosukKwon @miladm @shauheen
Google has made its efforts in supportin TPU. We will make efforts to utilize the canonical IR optimizattion ,auto tuning, auto fusion, layout optimization form xla and stableHLO in GPU devices.
Anything you want to discuss about vllm.
Hi, We have adapt our harware to pytorch/xla, what should we do to run vllm via pytorch/xla?