search

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Apache License 2.0
1.84k stars 173 forks source link

Can deepspeed-MII run on AMD GPU? #345

Open sunpian1 opened 9 months ago

sunpian1 commented 9 months ago

Can deepspeed-MII run on AMD GPU?

susie.sun@yz-amd1:~$ deepspeed --num_gpus 1 pipe.py [2023-12-05 09:42:21,499] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2023-12-05 09:42:21,780] [WARNING] [runner.py:203:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. Traceback (most recent call last): File "/home/susie.sun/.local/bin/deepspeed", line 6, in main() File "/home/susie.sun/.local/lib/python3.10/site-packages/deepspeed/launcher/runner.py", line 422, in main raise RuntimeError("Unable to proceed, no GPU resources available") RuntimeError: Unable to proceed, no GPU resources available

susie.sun@yz-amd1:~$ cat pipe.py import mii pipe = mii.pipeline("mistralai/Mistral-7B-v0.1") response = pipe(["DeepSpeed is", "Seattle is"], max_new_tokens=128) print(response)

mrwyattii commented 9 months ago

Hi @sunpian1, our custom kernels do not currently support AMD GPUs. It is something we have considered, and I will share this with the other devs. Thank you!

sunpian1 commented 9 months ago

@mrwyattii thanks for your answer!