-
**Description**
I'm experiencing segmentation fault while working with triton developer tools C++ API.
My code, running in multiple threads:
```
auto fut = server->AsyncInfer(*request);
auto resu…
-
Successfully ran inference with llama-2-7b. Can you confirm it's the llama2-7b-hf model that is pulled? From the logs it looks like it pulled that one from my cache.
Would the "chat" model not be b…
-
**Description**
When a model is unloaded via triton's unload API, it is observed that the other functions / APIs related to same model still works creating an improper API workflow.
```
1 http_serv…
-
`SequenceStates` objects have separate allocations for `input_states_` and `output_states_`. `output_states_` is written to and the `input_states_` is read from. After a batch is executed they are swa…
-
Hi Maintainers,
Firstly, thanks for the great work! I would like to deal with the following issue.
**Description**
When I enable Triton Server tracing in triton mode, a Request ID is part of a …
-
**Describe the bug**
A clear and concise description of what the bug is.
Hi, thank you for this fantastic app. I love it, and having ChatGPT, Bing, and Bard, an all-in-one search extension, is fan…
-
**Describe the bug**
Bing报错:ServiceClient failure for UserOffense
---> Failed to call "UserOffense" at "https://WestUS2.bing.prod.dlis.binginternal.com/route/wit.InappropriatenessClassifier_V1_0_7_…
-
**Description**
The command feature `reuse-http-port` to merge several triton servers in a single port is indicated boolean in the help flag, however it is actually implemented as integer.
**Trito…
-
Won't talk to bing/chat, it says:
ServiceClient failure for UserOffense
---> Failed to call "UserOffense" at "https://koreasouth.bing.prod.dlis.binginternal.com/route/wit.InappropriatenessClassifie…
-
**Is your feature request related to a problem? Please describe.**
This is a blocking issue for us.
gRPC-generated files are old and incompatible with many other libraries (tested for Python). As ha…
omidb updated
11 months ago