Open Fred-cell opened 10 months ago
the configuration is recommended, but not required.
and, after I set these configuration, the bug is still alive.
You can raise the system open file limit using
ulimit -n 2048
it is work around only, if other people encounter this issue, please append below.
Yes, it is a workaround, we will also monitor on our side if we encounter this in the future.
shall we add it to llm-init?
shall we add it to llm-init?
Maybe not? We will monitor if this issue happens in our env or by others to follow up this.
shall we add it to llm-init?
1024 is enough on our desktops. And I monitored the fd opened when running llama2-7b-half in fp8 precision 32-32 (20 times) on arc02, it shows this process opens up to 166 fds. We may need to find the root cause and decide whether to set this parameter. Since this setting will affect other applications running on the current terminal.
bigdl-core-xe 2.4.0b20231101 bigdl-core-xe-esimd 2.4.0b20231101 bigdl-llm 2.4.0b20231101
the second inference failed as "segmentation default" log as below: /root/anaconda3/envs/bigdl-llm/lib/python3.10/site-packages/bigdl/llm/transformers/models/chatglm2.py:137: UserWarning: IPEX XPU dedicated fusion passes are enabled in ScriptGraph non profiling execution mode. Please enable profiling execution mode to retrieve device guard. (Triggered internally at /build/intel-pytorch-extension/csrc/gpu/jit/fusion_pass.cpp:826.) query_layer = apply_rotary_pos_emb(query_layer, rotary_pos_emb) =========First token cost 5.1180 s========= =========Rest tokens cost average 0.0331 s (294 tokens in all)========= Segmentation fault (core dumped)