triton-inference-server / triton_cli

Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inference Server.
48 stars 2 forks source link

Fix vLLM profiler bug, add fallback logic to server start, cleanup #20

Closed rmccorm4 closed 9 months ago

rmccorm4 commented 9 months ago

Locally fixed and verified that "all-in-one" bench workflow, and individual subcommand workflows behave the same:

triton bench -m gpt2

and

triton repo clear
triton repo add -m gpt2
triton server start
triton model profile -m gpt2
fpetrini15 commented 9 months ago

LGTM, except for the nit and undefined server error. A lot of great adds here!