ViT onnx export

[bnabis93]

Objective

• ViT model export for serving

TODO

• [x] Simple ViT onnx export
• [ ] Custom (using code) ViT onnx export
• [x] Check Inference speed Transformer's ViT
• [x] Check Inference speed ONNX ViT
• [x] Compare the inference speed Transformer's ViT and ONNX ViT

[bnabis93]

Simple ViT onnx export

• If you already know model ID, you can export onnx model easily.
• Model ID : google/vit-base-patch16-224, https://huggingface.co/google/vit-base-patch16-224
• atol is the tolerance.
• Device : 2080ti
• torch model average inference time: 0.006316 seconds

  
  $ python -m transformers.onnx --model=google/vit-base-patch16-224 --atol 1e-3 onnx/

Measure inference time.

$ python inference.py ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'] 2023-04-05 01:24:35.075534148 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:515 CreateExecutionProviderInstance] Failed to create TensorrtExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#requirements to ensure all dependencies are met. Input name: pixel_values, output name: last_hidden_state Average Inference time: 0.006433 seconds