Onnx transformers: Quantize option

patil-suraj / onnx_transformers

Accelerated NLP pipelines for fast inference on CPU. Built with Transformers and ONNX runtime.

Apache License 2.0

125 stars 27 forks source link

Onnx transformers: Quantize option #6

Open yusufcakmakk opened 3 years ago

yusufcakmakk commented 3 years ago

I've made changes as we talked in pr.

I added option local_model to pipeline. It ignores modelcard to load local models that without having modelcard.

I kept framework as torch. In some cases like loading local models i have got error that InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type.. We can leave it like this to stay safe.