search

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Apache License 2.0
1.76k stars 163 forks source link

Always Flush UIDs after Exceptions #491

Closed weiqisun closed 4 days ago

weiqisun commented 1 month ago

This PR fixes the issue where the MII server hangs after encountering 9999 exceptions. This issue was initially reported in https://github.com/microsoft/DeepSpeed-MII/issues/489.