search

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Apache License 2.0
1.76k stars 163 forks source link

Tf32 support #481

Open Chasapas opened 1 month ago

Chasapas commented 1 month ago

Hello, i get this error when trying to start a server with tf32 model NotImplementedError: Only fp16 and bf16 are supported Any idea or workaround? Thank you