issues
search
ELS-RD
/
transformer-deploy
Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
https://els-rd.github.io/transformer-deploy/
Apache License 2.0
1.64k
stars
150
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
fix: fix authentication for private models
#133
ayoub-louati
closed
2 years ago
0
Failed to load private model
#132
Matthieu-Tinycoaching
closed
2 years ago
5
Feat/add t5 support
#131
ayoub-louati
closed
1 year ago
7
t5 notebook broken with transformer-deploy 0.5.0
#130
michaelroyzen
opened
2 years ago
3
[Question] Documentation for generative model API and parameters?
#129
tanmayb123
opened
2 years ago
1
[Question] Batch inference for classification models
#128
denisov-vlad
opened
2 years ago
3
Unable to pull docker image
#127
brevity2021
opened
2 years ago
1
TypeError: deserialize_cuda_engine(): incompatible function arguments.
#126
harishprabhala
closed
2 years ago
0
feat: add t5 model support for convert script
#125
ayoub-louati
closed
2 years ago
0
GPU quantization for sentence-transformer: ONNX quantized model
#124
Matthieu-Tinycoaching
opened
2 years ago
3
Question about generative model notebook
#123
hyunwoongko
opened
2 years ago
1
Support converting T5 model
#122
ayoub-louati
closed
1 year ago
4
How to Run With Polygraphy Graph Surgeon
#121
sam-h-bean
opened
2 years ago
5
ValueError: Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModelForSequenceClassification.
#120
harishprabhala
opened
2 years ago
3
[Question] TensorRT + ONNX Runtime
#119
MaximilienLeClei
closed
2 years ago
2
t5_bf16 notebooks fails with [ONNXRuntimeError] : 10 : INVALID_GRAPH
#118
michaelroyzen
opened
2 years ago
4
embedding task fails with batch_size > 1 given variable length inputs
#117
bbartlett-nv
closed
2 years ago
4
Fix/update Triton + ORT
#116
pommedeterresautee
closed
2 years ago
1
Cannot build TensorT engine for classification models
#115
CecileGiang
opened
2 years ago
2
feat: add torchdynamo notebook
#114
pommedeterresautee
closed
2 years ago
0
split model between host memory and GPU
#113
pommedeterresautee
opened
2 years ago
0
simple benchmark with torchdynamo
#112
pommedeterresautee
closed
2 years ago
0
HF pipeline based inference
#111
kamalkraj
opened
2 years ago
9
[ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (tensor(int64)) , expected: (tensor(int32))
#110
Matthieu-Tinycoaching
opened
2 years ago
3
Optimising ONNX Graph either takes too long or doesn't seem to work
#109
accountForIssues
opened
2 years ago
4
fix: make tests faster
#108
pommedeterresautee
closed
2 years ago
0
Pytorch 1.12 breaks GPT-2 (and probably T5)
#107
pommedeterresautee
closed
2 years ago
4
onnx optimized for distilbert fails with batch > 1
#106
kamalkraj
closed
2 years ago
1
fix: add docker image version warning in README
#105
pommedeterresautee
closed
2 years ago
0
Docker Hub image seems to not be synched with the main branch
#104
Thytu
closed
2 years ago
1
add transformers modifications to boost performance on inference
#103
ayoub-louati
closed
2 years ago
7
Optimizations for T0
#102
michaelroyzen
closed
2 years ago
10
Probabilities for classification task comparing to PyTorch
#101
denisov-vlad
closed
2 years ago
4
UnboundLocalError: local variable 'network_definition' referenced before assignment
#100
Ki6an
closed
2 years ago
3
feat: reduce memory footprint of T5 notebook to make it work with 3b model
#99
pommedeterresautee
closed
2 years ago
0
Doc: update 'Query the inference server' part in the run.md
#98
Thytu
closed
2 years ago
0
Error Nodes in a graph must be topologically sorted, however input 'encoder_hidden_states' of node: name: MatMul_173_input_cast0 OpType: Cast is not output of any previous nodes
#97
dheerajiiitv
opened
2 years ago
1
Test fastseq project: https://github.com/microsoft/fastseq
#96
ayoub-louati
closed
2 years ago
0
Switch T5 notebook to 3B model
#95
pommedeterresautee
closed
2 years ago
0
do we support converd electra model to ONNX or Tensorrt
#94
zhaohb
closed
2 years ago
3
fix: simplify NER pipeline on Triton server
#93
pommedeterresautee
closed
2 years ago
0
add text generation for t5 model with tensorrt runtime
#92
ayoub-louati
closed
2 years ago
0
fix: triton configuration refactoring
#91
gaetansnl
opened
2 years ago
0
fix: update triton to make it work with last ORT version
#90
pommedeterresautee
closed
2 years ago
0
Mixed precision conversion getting Assertion Error
#89
caffeinetoomuch
opened
2 years ago
15
Error in docker container using pip install
#88
lakshaykc
closed
2 years ago
2
Dynamic batching does not give better latency for Roberta running on TensorRT.
#87
Ki6an
closed
2 years ago
12
convert_model: error: argument --task: invalid choice: 'token-classification' (choose from 'classification', 'embedding', 'text-generation')
#86
Ki6an
closed
2 years ago
4
big performance difference on tensorRT
#85
HireezShanPeng
opened
2 years ago
4
Add question-answering model
#84
kamalkraj
closed
2 years ago
15
Previous
Next