Closed rifkybujana closed 1 year ago
I tried running it on different devices and using another model, but none worked.
I think the main problem is from
E0502 22:17:32.770347 1 model_repository_manager.cc:1355] failed to load 'transformer_onnx_model' version 1: Invalid argument: model 'transformer_onnx_model', tensor 'output': the model expects 2 dimensions (shape [-1,384]) but the model configuration specifies 2 dimensions (shape [-1,-1])
It is fixed by changing the output configuration in the model config (config.pbtxt
) from
output {
name: "output"
data_type: TYPE_FP32
dims: [-1, -1]
}
to
output {
name: "output"
data_type: TYPE_FP32
dims: [-1, 768]
}
it would be more convenient if this fix can be implemented directly on the convertion
Hi, I am trying to build a dense embedding inference using this repository. When I tried using the model on the example (
sentence-transformers/msmarco-distilbert-cos-v5
) it worked fine. However, using a different model, even the model from sentence-transformers itself, likesentence-transformers/all-MiniLM-L6-v2
, won't run the inference. Here are the full logs:I am using a CPU-only device to test it, although I have tried it on a GPU device which also doesn't work. Would you happen to have any idea on how to fix it? here's the command I use: