fixie-ai / ultravox

MIT License
618 stars 24 forks source link

Evaluate Ultravox performance when quantized #8

Open juberti opened 3 weeks ago

juberti commented 3 weeks ago

Quantize Ultravox to fp8 and determine how this affects the model's inference performance as well as speed. This would entail