issues
search
triton-inference-server
/
triton_cli
Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inference Server.
48
stars
2
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add note on MPI dependencies
#34
rmccorm4
closed
8 months ago
0
Give GPT2 quicker build/load settings for demos, fix Dockerfile version syntax, bump CLI version to 0.0.4
#33
rmccorm4
closed
8 months ago
0
Bump cli version to 0.0.3, bump trtllm version to 0.7.1, and bump vllm version to 0.3.0
#32
rmccorm4
closed
8 months ago
3
Bring back IFB default to TRT LLM models and bump to 24.01
#31
rmccorm4
closed
8 months ago
1
Minor Repo Optimizations
#30
fpetrini15
closed
8 months ago
0
Fix profile subcommand to account for offline (non-streaming) metrics and V1 batching
#29
rmccorm4
closed
8 months ago
0
Fix model infer on TRT LLM with negative ints, and minor cleanup
#28
rmccorm4
closed
8 months ago
0
Add --backend support to bench command and default to custom image
#27
rmccorm4
closed
8 months ago
0
Modularize TRT LLM Builders
#26
fpetrini15
closed
8 months ago
0
Fix --prompt for different shapes, ignore onnx files on HF download, conditional import
#25
rmccorm4
closed
8 months ago
0
GPT Engine Builder
#24
fpetrini15
closed
8 months ago
1
Catch errors and improve logging in Profiler
#23
nv-hwoo
closed
9 months ago
0
Bump version to 0.0.2
#22
rmccorm4
closed
8 months ago
0
Add initial tests for repo subcommand
#21
rmccorm4
closed
9 months ago
1
Fix vLLM profiler bug, add fallback logic to server start, cleanup
#20
rmccorm4
closed
9 months ago
1
Add copyrights and minor cleanup
#19
rmccorm4
closed
9 months ago
0
Automatic TRT LLM Detail Parsing
#18
fpetrini15
closed
9 months ago
0
Add demo features for benchmarking LLMs
#17
rmccorm4
closed
9 months ago
1
Fix high concurrency generation throughput calculation
#16
nv-hwoo
closed
10 months ago
0
POC: Background Server
#15
fpetrini15
closed
10 months ago
0
Misc fixes
#14
rmccorm4
closed
10 months ago
1
Add profile subcommand to run perf analyzer
#13
matthewkotila
closed
10 months ago
0
Variable name change for clarity
#12
oandreeva-nv
closed
10 months ago
0
Minor TRT-LLM tweaks
#11
rmccorm4
closed
10 months ago
0
Kyang update script
#10
jbkyang-nvi
closed
10 months ago
0
Kyang update script
#9
oandreeva-nv
closed
10 months ago
0
Basic MPI support
#8
fpetrini15
closed
10 months ago
2
Populate model repo with TRTLLM templates
#7
oandreeva-nv
closed
10 months ago
1
Add rough NGC CLI wrapper
#6
rmccorm4
closed
10 months ago
0
Add README and update default image
#5
rmccorm4
closed
10 months ago
0
Add initial prototype
#4
rmccorm4
closed
10 months ago
0
Prototype tool with simple client, repo, and server features
#3
rmccorm4
closed
10 months ago
0
Add pre-commit hook to upgrade Python syntax
#2
dyastremsky
closed
11 months ago
2
Setup repo and package structure
#1
rmccorm4
closed
11 months ago
0
Previous