-
https://github.com/triton-inference-server/redis_cache/blob/437e7e214ff446e6bc12febb44872547a1988fe1/src/redis_cache.cc#L220
Triton inference server is failing to write to Redis on versions past 4.…
-
Hello,
I've been playing around with the collab examples on both AMD and NVDA sides and noticed that when using Triton's latest (main branch) it seems to break down at Unsloth's init step in the __…
-
Hello, this is nice work of Triton Server, I would like to ask how to load/unload model in explicit mode of triton? Do you have any relevant code or ideas? Looking forward to your reply.
-
@Tabrizian In [this](https://github.com/triton-inference-server/client/blob/main/src/python/examples/simple_grpc_infer_client.py) example (line 131) they pass a python dict in the `headers` arg of `t…
-
WARNING: Num classes mismatch. Configured: 80, detected by network: 0
python3: nvdsparsebbox_Yolo.cpp:137: bool NvDsInferParseCustomYolo(const std::vector&, const NvDsInferNetworkInfo&, const NvDsInf…
-
### System Info
V100*2
nvcr.io/nvidia/tritonserver:24.01-trtllm-python-py3
tensorrt-llm 0.7.0
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [ ] My own mo…
-
I am testing on the basic models. Model take input and return the same output of same datatype.
Inference is happening:
2024-08-20 09:35:15,923 - INFO - array_final: array([[103]], dtype=uint8)
a…
-
**Description**
Triton crashes during runtime。
```
(gdb) info stack
#0 0x00007ffff64e4d8b in _int_malloc (av=av@entry=0x7ffe30000020, bytes=bytes@entry=24) at malloc.c:3608
#1 0x00007ffff64e729…
-
**Description**
PR [185](https://github.com/triton-inference-server/client/pull/185) pinned `geventhttpclient==2.0.2` due to a potential change in ssl_context_factory handling.
The geventhttpcli…
-
I have nvidia and nvidia-docker installed. I have a 1060 with 6GB vram it has compute 6.0
how can I troubleshoot this? when I run other nvidia containers on my PC I have to use --privileged to get …