This PR introduces OpenVINO Runtime support in Spark NLP
Description
This PR enables Spark NLP to leverage the OpenVINO Runtime API for Java to load and run models in various formats including ONNX, PaddlePaddle, Tensorflow, Tensorflow Lite and OpenVINO IR format. OpenVINO also enables performance improvements when running on supported intel hardware, with upto 40% improvement vs Tensorflow on benchmarks with no further tuning. You can also take advantage of the full optimization and quantization capabilities offered by the OpenVINO toolkit when exporting/converting the model to the OpenVINO format using the Model Conversion API.
The following annotators have been enabled to work with OpenVINO:
Note: To take advantage of this feature, see these instructions to build OpenVINO jar (Linux), and these to build Spark NLP. OpenVINO is cross-platform. Refer here for Windows build instructions, and here for other platforms.
Motivation and Context
Out-of-the-box optimizations and better performance on supported Intel hardware
Capable of reading ONNX, PaddlePaddle, TensorFlow and TensorFlow Lite formats directly
This work was completed as part of Google Summer of Code 2023
Screenshots (if appropriate):
Types of changes
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Code improvements with no or little impact
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
This PR introduces OpenVINO Runtime support in Spark NLP
Description
This PR enables Spark NLP to leverage the OpenVINO Runtime API for Java to load and run models in various formats including ONNX, PaddlePaddle, Tensorflow, Tensorflow Lite and OpenVINO IR format. OpenVINO also enables performance improvements when running on supported intel hardware, with upto 40% improvement vs Tensorflow on benchmarks with no further tuning. You can also take advantage of the full optimization and quantization capabilities offered by the OpenVINO toolkit when exporting/converting the model to the OpenVINO format using the Model Conversion API.
The following annotators have been enabled to work with OpenVINO:
Note: To take advantage of this feature, see these instructions to build OpenVINO jar (Linux), and these to build Spark NLP. OpenVINO is cross-platform. Refer here for Windows build instructions, and here for other platforms.
Motivation and Context
Screenshots (if appropriate):
Types of changes
Checklist: