Add bigdl-llm loader to bigdl-upstream

[chtanch]

Description:

Add BigDL-LLM loader to upstream main

• Core changes / functionality only
• No GPU support
• Readme to be updated after scope of commit is reviewed

Environment setup

pip install --pre --upgrade bigdl-llm[all] cd \<text-generation-webui> pip install -r requirements_cpu_only.txt

Run WebUI

python server.py --load-in-low-bit sym_int4 --share

Note:

• --share is required if program is run on remote server through ssh. If this option is not enabled, an error is thrown.

Tested models

Successful - chatglm3-6b, mistral-7B-v0.1, qwen-7b-chat, falcon-7b-instruct-with-patch Fail - llama2-7b-chat-hf. Reason is upstream main's requirements*.txt has updated transformers version to 4.38.2, which is unsupported.

[shane-huang]

update bigdl-llm to ipex-llm.

[hkvision]

@jason-dai @shane-huang The current version as the first commit for upstream seems good, merge it first.