triton-inference-server client issues

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

BSD 3-Clause "New" or "Revised" License

516 stars 224 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Delayed aio infer during burst requests

#733 evanarlian opened 7 hours ago
0
Guard GenAI-Perf plot generation

#732 dyastremsky closed 17 hours ago
1
Memory leak from grpcio

#731 AlexanderKomarov opened 1 day ago
0
Can not import GRPC Tritonclient in Seldon MLServer

#730 haiminh2001 closed 1 day ago
1
Fix filepath

#729 debermudez closed 4 days ago
0
Add support for Hugging Face Text Embeddings Interface's re-ranker API

#728 dyastremsky opened 4 days ago
0
Add testing for DataLoader

#727 AndyDai-nv opened 4 days ago
2
Bug in grpc client

#726 gerasim13 opened 4 days ago
0
Update tutorial to be two commands each

#725 tgerdesnv opened 4 days ago
0
Add rankings support

#724 debermudez closed 4 days ago
1
Support ranking API profile data parsing and metrics calculation

#723 nv-hwoo closed 4 days ago
1
Minimum Impact PA Migration Changes

#722 fpetrini15 opened 5 days ago
0
Update GenAI-Perf metric unit assignment to avoid overwrites

#721 dyastremsky closed 5 days ago
0
[WIP] LLaVA support

#720 mwawrzos opened 6 days ago
0
Support embedding metrics and output export

#719 nv-hwoo closed 5 days ago
1
Update GenAI-Perf development version

#718 mc-nv closed 5 days ago
0
Document how to profile embeddings models

#717 dyastremsky closed 4 days ago
0
Migrate PA repo

#716 fpetrini15 opened 6 days ago
0
Add embedding and ranking support

#715 dyastremsky closed 4 days ago
0
How to write cmakelist to use grpcclient?

#714 lyp741 opened 1 week ago
0
Add embedding support

#713 dyastremsky closed 6 days ago
0
Reorganize metrics and data parser

#712 nv-hwoo closed 1 week ago
2
[QUESTION] - Tensorflow python infer to triton client

#711 eyalhir74 opened 1 week ago
0
tritonclient.utils.InferenceServerException: [StatusCode.INTERNAL] Unable to open shared memory region: '/output_simple'

#710 tangxueduo opened 2 weeks ago
0
Generate LLM inputs for embeddings endpoint

#709 dyastremsky closed 1 week ago
1
Support embeddings and rankings in GenAI-Perf

#708 dyastremsky closed 2 weeks ago
1
Piotrm request client

#707 piotrm-nvidia opened 2 weeks ago
0
Remove ITL from console when non-streaming

#706 nv-hwoo closed 2 weeks ago
1
Update input data docs to include allowing outputs in provided directory

#705 matthewkotila closed 1 week ago
0
Fix tutorials

#704 tgerdesnv closed 2 weeks ago
0
genai-perf KeyError: 'service_kind'

#703 highheart closed 15 hours ago
6
Allow multiple prompts to be supplied via --input-file to GenAI-Perf

#702 dyastremsky closed 2 weeks ago
3
Update metric name from input/output tokens to input/output sequence lengths

#701 nv-hwoo closed 2 weeks ago
4
Add tensorrtllm_engine option to service-kind and update testing

#700 debermudez closed 2 weeks ago
0
How do I get genai-perf to analyze my defined data set

#699 highheart closed 15 hours ago
2
Calculate total output token from full text

#698 IzzyPutterman closed 2 weeks ago
4
fix: Get validation outputs by name rather than index

#697 krishung5 closed 3 weeks ago
2
Add documentation on installing PA dependencies on Ubuntu

#696 matthewkotila closed 3 weeks ago
0
Clamp window for request-count

#695 tgerdesnv closed 4 days ago
0
Update ITL calculation

#694 nv-hwoo closed 3 weeks ago
0
Add code to catch error at line 58

#693 lkomali closed 3 weeks ago
0
Add a small line to catch error at line 58

#692 lkomali closed 3 weeks ago
0
Fix HTTP client REQUEST_END timestamp

#691 dyastremsky closed 3 weeks ago
0
ci: Restrict numpy to version 1.x

#690 KrishnanPrash closed 3 weeks ago
0
Document artifact-dir arg in README

#689 dyastremsky closed 3 weeks ago
0
Only require throughput stability for PA HTTP async case

#688 dyastremsky closed 3 weeks ago
9
Fix typo

#687 matthewkotila closed 3 weeks ago
1
Fix HTTP Client Async Code for Request Rate

#686 dyastremsky closed 3 weeks ago
1
fix: Get validation outputs by name rather than index

#685 rmccorm4 closed 3 weeks ago
1
fix: Fix Client Http Async Code for Request Rate

#684 nnshah1 closed 2 weeks ago
0