[ML] Add Intel PyTorch Extension support on linux-x86_64

elastic / ml-cpp

Machine learning C++ code

Other

7 stars 62 forks source link

[ML] Add Intel PyTorch Extension support on linux-x86_64 #2547

Closed valeriy42 closed 1 year ago

valeriy42 commented 1 year ago

This PR integrates the Intel Extension for PyTorch (IPEX) into our build process so that PyTorch models quantized using the IPEX backend can be evaluated using the pytorch_inference processor. IPEX is currently only supported on Linux machines with x86_64 architecture.

Our research has shown that the main benefit of IPEX quantization comes from the use of the BFloat16 data type. Therefore, the pytorch inference processor has been extended to support this data type as well.

valeriy42 commented 1 year ago

@droberts195, thank you for the review. It would be great if you could give it another pass.

P.S. Build kite is failing because I didn't upload the docker container yet.