Jasonzzt
commented
1 year ago Issue1 on cpu with python 3.10
On spr-01, I followed all-in-one to run bigdl-llm on Intel CPU with python3.10.
conda create --name ziteng-310 python=3.10
conda activate ziteng-310
pip install omegaconf
pip install pandas
pip install --pre --upgrade bigdl-llm[all]
pip install bigdl-nano[pytorch]
source bigdl-nano-initBy running the above command, I got some conda environment configuration which are as follows
python 3.10.13
bigdl-llm 2.4.0b20231024
bigdl-nano 2.3.0I tested the models chatglm2-6b, Llama-2-7b-chat-hf, Baichuan2-7B-Chat, Llama-2-13b-chat-hf and mpt-7b-chat. Except for the chatglm2-6b model, all other models are fine to run bigdl-llm on cpu with python3.10.
When running bigdl-llm on cpu with chatglm2-6b, I got the following problem
Traceback (most recent call last):
File "/root/ziteng/BigDL/python/llm/dev/benchmark/all-in-one/./run.py", line 552, in <module>
run_model(model, api, conf['in_out_pairs'], conf['local_model_hub'], conf['warm_up'], conf['num_trials'], conf['num_beams'], conf['low_bit'])
File "/root/ziteng/BigDL/python/llm/dev/benchmark/all-in-one/./run.py", line 45, in run_model
result = run_transformer_int4(repo_id, local_model_hub, in_out_pairs, warm_up, num_trials, num_beams, low_bit)
File "/root/ziteng/BigDL/python/llm/dev/benchmark/all-in-one/./run.py", line 144, in run_transformer_int4
model = AutoModelForCausalLM.from_pretrained(model_path, load_in_low_bit=low_bit, trust_remote_code=True,
File "/root/anaconda3/envs/ziteng-310/lib/python3.10/site-packages/bigdl/llm/transformers/model.py", line 97, in from_pretrained
model = cls.load_convert(q_k, optimize_model, *args, **kwargs)
File "/root/anaconda3/envs/ziteng-310/lib/python3.10/site-packages/bigdl/llm/transformers/model.py", line 120, in load_convert
model = cls.HF_Model.from_pretrained(*args, **kwargs)
File "/root/anaconda3/envs/ziteng-310/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 496, in from_pretrained
raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers_modules.chatglm2-6b.configuration_chatglm.ChatGLMConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig, ErnieConfig, FalconConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MusicgenConfig, MvpConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, TransfoXLConfig, TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig.Running on version 3.9 also encountered this issue. It seemed that this was a problem with the model itself. A related problem was found in Chatglm's issue. https://github.com/THUDM/ChatGLM-6B/issues/37#issuecomment-1704036282 The trainer of huggingface tends to save only the model rather than both model with tokenizer.
Aside from this issue, There is no other problem running bigdl-llm on cpu with python 3.10.
liu-shaojun
Zephyr596
Issue1 on xpu with python 3.10 [Fixed after releasing bigdl-core-xe and bigdl-core-xe-esimd for python 3.10]
on Arc14, I followed https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/GPU/HF-Transformers-AutoModels/Model/llama2/README.md to run example with BigDL-LLM on Intel GPUs
bigdl-llm==2.4.0b20230810 is installed, it is weird that bigdl-core-xe and bigdl-core-xe-esimd are missing when python=3.10.
Then set env and run generate.py
I got the following error
After
pip install protobufandpython ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATHI got the following errorThere is a transformers issue https://github.com/huggingface/transformers/issues/26755 related to this error, should we use a fixed specific transformers version?
Solution: https://github.com/intel-analytics/llm.cpp/pull/128