-
I have trained some models in Tensorflow 2.16 / Keras 3 and inference works in a normal python process. It appears to load successfully in Triton but fails upon inference with the following error logs…
-
# Issue Description
Got empty result using Python Backend.
While the FauxPilet console shows error message
```
fauxpilot-copilot_proxy-1 | [StatusCode.UNAVAILABLE] failed to connect to all …
-
I noticed that there is no logging system in the Triton Python source code. While using print statements is easy, it is not suitable for a production environment. Is it possible for me to add a logger…
-
**Edited title: removed "[Pallas GPU]"**
### Edited Description
```python
import jax
import jax.numpy as jnp
jax.config.update('jax_numpy_dtype_promotion', 'strict')
x = jnp.array(2, dty…
-
Hi, is there any guide how to implement Yolo v4 TAO model into Triton inference server? I have trained Yolo v4 custom data model via TAO toolkit and looking for an guide how to implement this model wi…
-
# Background:
in the performance doc [https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/performance.md](url) mentioned:
LLama7B , FP16 , batchsize:256 , input_len:128 output_len:128…
-
During the process of distributed training, I encountered the following problem when compiling Triton kernels:
```
Traceback (most recent call last):
......
File "/mnt/petrelfs/caoweihan/anaconda3…
-
`triton ssh` looks up the image to see if it should use the ubuntu user for ssh instead of root. If the image has been disabled, or otherwise can't be looked up, `triton ssh` fails outright.
Instea…
-
# pip3 install triton
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement triton (from versions: none)
ERROR…
-
Hi friends:
I installed triton from the source. But the triton compiler generates an error which is "AttributeError: module 'triton' has no attribute 'jit'". Someone knows how to solve this? Than…