mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.08k stars 1.56k forks source link

[Bug] subprocess.CalledProcessError: Command '['D:\\Program\\Anaconda3\\envs\\mlc-chat\\python.exe', '-m', 'mlc_chat.cli.check_device', 'vulkan:0']' returned non-zero exit status 3221225477. #1323

Closed tao-began closed 11 months ago

tao-began commented 11 months ago

🐛 Bug

I used mlc llm to compile Llama-2-7b-chat-hf, but when using mlc-chat, I reported an error: subprocess.CalledProcessError: Command '['D:\Program\Anaconda3\envs\mlc-chat\python.exe', '-m', 'mlc_chat.cli.check_device', 'vulkan:0']' returned non-zero exit status 3221225477.

To Reproduce

Steps to reproduce the behavior: python sample_mlc_chat.py

Complete error reporting:

Traceback (most recent call last): File "E:\code\mlc-llm-win\mlc-llm\sample_mlc_chat.py", line 8, in cm = ChatModule(model="Llama-2-7b-chat-hf-q4f16_1") File "D:\Program\Anaconda3\envs\mlc-chat\lib\site-packages\mlc_chat\chat_module.py", line 660, in init self.device = detect_device(device) File "D:\Program\Anaconda3\envs\mlc-chat\lib\site-packages\mlc_chat\support\auto_device.py", line 27, in detect_device if _device_exists(cur_device): File "D:\Program\Anaconda3\envs\mlc-chat\lib\site-packages\mlc_chat\support\auto_device.py", line 58, in _device_exists result = subprocess.check_output(cmd, stderr=subprocess.STDOUT).decode("utf-8") File "D:\Program\Anaconda3\envs\mlc-chat\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "D:\Program\Anaconda3\envs\mlc-chat\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['D:\Program\Anaconda3\envs\mlc-chat\python.exe', '-m', 'mlc_chat.cli.check_device', 'vulkan:0']' returned non-zero exit status 3221225477.

Environment

Additional context

E:\code\mlc-llm-win\mlc-llm\dist\Llama-2-7b-chat-hf-q4f16_1 的目录

2023/11/23 16:16

. 2023/11/23 16:16 .. 2023/11/23 16:16 18,516,480 Llama-2-7b-chat-hf-q4f16_1-vulkan.dll 2023/11/23 16:16 7,387 Llama-2-7b-chat-hf-q4f16_1-vulkan.exp 2023/11/23 16:16 14,310 Llama-2-7b-chat-hf-q4f16_1-vulkan.lib 2023/11/23 16:16 30,528,875 mod_cache_before_build.pkl 2023/11/23 16:15 params

E:\code\mlc-llm-win\mlc-llm\dist\Llama-2-7b-chat-hf-q4f16_1\params 的目录

2023/11/23 16:15

. 2023/11/23 16:15 .. 2023/11/23 16:15 21 added_tokens.json 2023/11/23 16:15 629 mlc-chat-config.json 2023/11/23 16:15 142,309 ndarray-cache.json 2023/11/23 16:14 65,536,000 params_shard_0.bin


2023/11/23 16:15 1,842,767 tokenizer.json 2023/11/23 16:15 499,723 tokenizer.model 2023/11/23 16:15 770 tokenizer_config.json

junrushao commented 11 months ago

Could you run the following command and share the error message:

python -m mlc_chat.cli.check_device vulkan:0

tao-began commented 11 months ago

Could you run the following command and share the error message:

python -m mlc_chat.cli.check_device vulkan:0

python -m mlc_chat.cli.check_device vulkan:0,the result is 1

tao-began commented 11 months ago

Is it possible that this is because I am a GTX 1650 4G graphics card, because MLC currently does not support the CPU, and then the PC's graphics card is not enough memory?

MaTwickenham commented 11 months ago

My device is 3060TI 8G and I also encountered this problem.

Is it possible that this is because I am a GTX 1650 4G graphics card, because MLC currently does not support the CPU, and then the PC's graphics card is not enough memory?

junrushao commented 11 months ago

It’s not about GPUs, but more like an OS issue. @Tao-begd @MaTwickenham Are you both on windows?

Could you run the following command and share the error message: python -m mlc_chat.cli.check_device vulkan:0

python -m mlc_chat.cli.check_device vulkan:0,the result is 1

It seems to actually work…Did it crash or any error message popped up?

MaTwickenham commented 11 months ago

It’s not about GPUs, but more like an OS issue. @Tao-begd @MaTwickenham Are you both on windows?

Could you run the following command and share the error message: python -m mlc_chat.cli.check_device vulkan:0

python -m mlc_chat.cli.check_device vulkan:0,the result is 1

It seems to actually work…Did it crash or any error message popped up?

Yes, I encountered that error on Windows. And my error message is just like @Tao-begd posted.

tqchen commented 11 months ago

Seems that the subprocess checking is a bit unreliable in windows. Maybe we can return to vulkan().exist check in the same process ?

tqchen commented 11 months ago

https://github.com/mlc-ai/mlc-llm/pull/1333 may address this issue

tao-began commented 11 months ago

Thanks to Professor Tianqi, the problem is solved

tqchen commented 11 months ago

Thank you for reporting!

junrushao commented 11 months ago

I'm able to reproduce this issue:

image

using:

from mlc_chat.support.auto_device import detect_device
print(detect_device("auto"))

Removing "vulkan" in AUTO_DETECT_DEVICES avoids the sub-process from crashing, which means it's a TVM-specific bug with its Vulkan runtime @tqchen

junrushao commented 11 months ago

In fact, directly runs the command below actually does fail, but cmd.exe just hides this problem silently:

image

junrushao commented 11 months ago

https://github.com/mlc-ai/mlc-llm/pull/1350