zhaohb
commented
3 years ago I test with tritonclient and get the same error
Closed zhaohb closed 3 years ago
zhaohb
commented
3 years ago I test with tritonclient and get the same error
zlsh80826
commented
3 years ago Hello @zhaohb,
Is rec your own model? Can you share the config.pbtxt for more information?
zhaohb
commented
3 years ago config.pbtxt :
name: "rec"
backend: "paddle"
max_batch_size: 4096
input {
name: "x"
data_type: TYPE_FP32
dims: 3
dims: 32
dims: -1
}
output {
name: "save_infer_model/scale_0.tmp_1"
data_type: TYPE_FP32
dims: -1
dims: 6625
}
instance_group {
count: 1
kind: KIND_GPU
}
dynamic_batching {
preferred_batch_size: 1024
preferred_batch_size: 2048
max_queue_delay_microseconds: 100
}rec is also not my model, I get from https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.1/doc/doc_ch/inference.md
zhaohb
commented
3 years ago @zlsh80826 did you repeat the question？how to solve?
zlsh80826
commented
3 years ago @zhaohb , thank you for reaching out. I will look into this model these days.
zlsh80826
commented
3 years ago @zhaohb, which model you were using? There are many models in your provided link. The problem you met was that your provided output name can't be found in predictor_->GetOutputNames. For example, the output name of ch_ppocr_mobile_v2.0_rec_infer should be ctc_fc.tmp_1 instead of save_infer_model/scale_0.tmp_1.
Following is a workable config.pbtxt for ch_ppocr_mobile_v2.0_rec_infer
name: "rec"
backend: "paddle"
max_batch_size: 4096
input {
name: "x"
data_type: TYPE_FP32
dims: 3
dims: 32
dims: 100
}
output {
name: "ctc_fc.tmp_1"
data_type: TYPE_FP32
dims: 25
dims: 6625
}
instance_group {
count: 1
kind: KIND_GPU
}
dynamic_batching {
preferred_batch_size: 1024
preferred_batch_size: 2048
max_queue_delay_microseconds: 100
}@zlsh80826 I just use rec model https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.1/doc/doc_ch/inference.md#%E8%AF%86%E5%88%AB%E6%A8%A1%E5%9E%8B%E8%BD%ACinference%E6%A8%A1%E5%9E%8B
but when I open this model with netron, I can found the output is save_infer_model/scale_0.tmp_1:
So, where does this method come from, and how do I call it to get the name of the output?
zlsh80826
commented
3 years ago You can add the following lines
output_names = self.predictor.get_output_names()
print(output_names)at https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.1/tools/infer/predict_rec.py#L237
Then follow the PaddleOCR instruction to run prediction and print the output names. e.g. python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words/ch/word_4.jpg" --rec_model_dir inference/rec_crnn
Why the output tensor is not the last layer showed in Netron is out of my knowledge, you might ask on PaddlePaddle. But I guess maybe the op fusion strategy affects it.
zhaohb
commented
3 years ago ok, thank you very much.
zlsh80826
commented
3 years ago Solved
I compiled the backend of paddle using version 21.04 and successfully generated the libtriton_paddle.so required by triton and successfully loaded the model of paddle, but I made an error when I pressed the model using perf_Analyzer:
triton outout:
How to solve this problem？