issues
search
triton-inference-server
/
fastertransformer_backend
BSD 3-Clause "New" or "Revised" License
411
stars
133
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Build backend inside the docker container, undefined symbol
#120
A-ML-ER
opened
1 year ago
4
model output must specify 'data_type' for fastertransformer
#119
18810251126
opened
1 year ago
6
Does triton-inference-server only support slurm for multi-node deployment?
#118
Shuai-Xie
opened
1 year ago
3
Question on how to set --shape when using perf_analyzer
#117
YJHMITWEB
opened
1 year ago
3
Any support plan for VisionEncoderDecoderModel?
#116
kangsan0420
opened
1 year ago
1
compile my own backend, libtriton_fastertransformer.so undefined symbol:
#115
A-ML-ER
opened
1 year ago
7
All flan-t5 doesn't work for me
#114
PetroMaslov
opened
1 year ago
3
Questions about different intra-node settings for fastertransformer_backend and FasterTransformer
#113
YJHMITWEB
opened
1 year ago
4
Questions about model instances and dynamic batch when setting model concurrency
#112
YJHMITWEB
closed
1 year ago
2
CUDA runtime error: CUDA driver version is insufficient for CUDA runtime version on FT
#111
lkm2835
opened
1 year ago
1
docs: fix README.md
#110
lkm2835
opened
1 year ago
0
When I compile ft_backend based on cuda10.2 report nvcc fatal : Unsupported gpu architecture 'compute_80'
#109
nercoeus
closed
1 year ago
2
could end_to_end_test.py with model_name 'ensemble' support decoupled mode
#108
jimmyforrest
opened
1 year ago
6
Support dockerhub push
#107
wsxiaoys
closed
1 year ago
0
Support dockerhub push
#106
wsxiaoys
closed
1 year ago
0
E0315 1107 server.cc:201] Failed to finalize CUDA memory manager: CNMEM_STATUS_CUDA_ERROR
#103
WangYizhang01
opened
1 year ago
0
I don't know the cause of this error.
#102
amazingkmy
closed
1 year ago
8
CUDA architecture ignored when passed to Cmake
#101
hillct
opened
1 year ago
5
CPU maxed out, no GPU utilization, inference never completing
#100
zoltan-fedor
closed
1 year ago
1
gptj: fix abysmally slow postprocessor performance; don't read a file for each new batch
#99
git-bruh
closed
1 year ago
0
cuda function architecture error when trying to query the triton server .
#98
gd1m3y
closed
1 year ago
4
triton server crashed after reload the same model
#97
heiruwu
opened
1 year ago
2
Flan-T5 quality decreases with bigger models when using fastertransformer
#95
lakshaykc
opened
1 year ago
10
Some docker build fixes
#94
tanmayv25
closed
1 year ago
2
repo fails to build using Triton Image 23.01
#93
Chris113113
opened
1 year ago
2
Use the Triton client to call the interface asynchronously and call the interface concurrently, and the result is empty
#92
PAOPAO6
closed
1 year ago
2
GPT-J streaming: getting garbage response
#91
vax-dev
opened
1 year ago
1
Dynamic batching is not working for gptj
#90
PoodleWang
closed
1 year ago
2
fix error handling
#89
rr0gi
closed
1 year ago
1
Getting empty response from GPT-J Model
#88
vax-dev
opened
1 year ago
8
Serving large models with FT backend keeps Triton server crashing and restarting
#86
RajeshThallam
opened
1 year ago
0
Ragged Batching on Megatron Fast Transformer Backend
#85
mshuffett
opened
1 year ago
4
feat: update v1.4
#84
byshiue
closed
1 year ago
0
Create multistage build script for docker build
#83
jbkyang-nvi
closed
1 year ago
0
Supporting for Flan-t5 with gated activation and non-shared embeddings
#82
LydiaXiaohongLi
closed
1 year ago
3
T5 cross_attention output cannot be accessed
#81
JustinAWei
opened
1 year ago
1
Not getting response with warning "response is nullptr"
#80
t13m
opened
1 year ago
1
How can I get the logits of all tokens in vocab at each step?
#79
kevinlee819
opened
1 year ago
6
After triton fastertransformer backend, the inference speed is severely reduced
#78
PAOPAO6
closed
1 year ago
34
server crashs when traffic is a little bit high
#77
rahuan
opened
1 year ago
10
How much VRAM BLOOM consumes?
#76
pai4451
opened
1 year ago
6
feat: update v1.3 codes
#75
byshiue
closed
1 year ago
0
[ERROR] Does not find the section encoder with name relative_attention_num_buckets_or_max_pos_seq_len
#74
520jefferson
closed
1 year ago
17
Config.pbtxt for all_models/t5/fastertransformer incorrect
#73
dhaval24
opened
1 year ago
1
dose support have many same model instance in one GPU device?
#72
changleilei
closed
1 year ago
5
Support BLOOM model?
#69
pai4451
closed
1 year ago
4
did fastertransformer support version nvcr.io/nvidia/tritonserver:21.07-py3
#68
changleilei
closed
1 year ago
2
How to support different models with different tensor_para_size?
#67
TopIdiot
opened
1 year ago
29
T5: Triton Model Repository (containing model weights and configuration) on S3 doesn't work as expected
#66
dhaval24
opened
1 year ago
5
T5 not performing as expeceted
#64
nrakltx
opened
1 year ago
3
Previous
Next