Is the DeepSpeed-MII will support habana (HPU) hardware?

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

Apache License 2.0

1.77k stars 166 forks source link

Is the DeepSpeed-MII will support habana (HPU) hardware? #416

Open muhammad-asn opened 5 months ago

muhammad-asn commented 5 months ago

Currently the TGI have tgi-gaudi to support the HPU for Gaudi 2 and also VLLM is included their roadmap in Q1 2024

Is the DeepSpeed-MII will support habana (HPU) hardware as well?

Thank you

### Tasks

mrwyattii commented 5 months ago

We are currently working on support for HPU hardware on the training side of DeepSpeed. I expect we will also work towards supporting it for inference as well, but we do not have any set goals/deadlines around that support at this time.

muhammad-asn commented 4 months ago

@mrwyattii thanks