Integrate ITREX to support int8 model on the CPU-only devices

[yiliu30]

1st PR for https://github.com/stochasticai/xTuring/issues/264 to integrate Intel-Extension-for-Transformers to support int8 model on the CPU-only devices.

Usage

from xturing.models import BaseModel

# Auto detect the device and load int8 with Itrex if cpu-only
model = BaseModel.create("llama2_int8")

# Inference model with Itrex's highly optimized kernels
output = model.generate(texts=["Why are the LLM models important?"])
print(output)

TODO

• [x] Load model with Itrex for CPU-only device
• [x] Add example (gpt2, llama2)
• [ ] Add UT (WIP)
• [ ] Doc

@StochasticRomanAgeev

[yiliu30]

Hi @StochasticRomanAgeev, could you please take the time to conduct a preliminary review? thanks :)

[yiliu30]

Hi @StochasticRomanAgeev @tushar2407 @MarcosRiveraMartinez, could you please take your time to review the PR, thanks :)

[StochasticRomanAgeev]

Hi @yiliu30, Done merging, thanks for doing this integration!

[StochasticRomanAgeev]

It would be great if you could include a brief description of update into CPU inference section in the README. I intend to provide this enhancement in a subsequent PR.

[yiliu30]

It would be great if you could include a brief description of update into CPU inference section in the README. I intend to provide this enhancement in a subsequent PR.

Sure, I'll add an introduction in a follow-up PR soon :)