search

CentML / flexible-inference-bench

A modular, extensible LLM inference benchmarking framework that supports multiple benchmarking frameworks and paradigms.
Apache License 2.0
5 stars 0 forks source link

[API] Add ignore eos and some cleanup #43

Closed andoorve closed 5 months ago

andoorve commented 5 months ago

Adds ignore eos for openai endpoints and cserve debug.