Segmentation fault after change cuda-based python code to xpu-based python code with codegeex2-6b mode

[ganghe]

Hi Team,

I tried to change codegeex2 cuda-based python example code to xpu-based python code with codegeex2-6b mode file, but the program always crashed.

The cuda-based python file is here, https://github.com/THUDM/CodeGeeX2/blob/main/demo/fastapicpu.py

I can run it with cpu device, e.g. "python fastapicpu.py --model-path codegeex2-6b --cpu". but, I changed cuda-based python code to xpu-based python code, run it with gpu device with the command as below source /opt/intel/oneapi/setvars.sh --force python -X faulthandler fastapicpu.py --model-path codegeex2-6b

Then, the program crashed at line 207, "model = model.to('xpu')"

The related files are attached. fastapicpu.old.txt - cuda based python file fastapicpu.py.txt - xpu based python file cmd.txt - startup command script output.txt - program output log mode file - https://hf-mirror.com/THUDM/codegeex2-6b

cmd.txt fastapicpu.old.txt fastapicpu.py.txt output.txt

[NovTi]

Thank you for your question. This problem has been reproduced and I am currently working on it

[NovTi]

This issue is caused by the module import order and we suggest you put import torch above from transformers import AutoTokenizer's and from bigdl.llm.transformers import AutoModel like this to solve this error.

import torch
from transformers import AutoTokenizer
from bigdl.llm.transformers import AutoModel

[ganghe]

Hi NovTi,

Thank for your helps. The fix works for me.

-Gang