search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.26k stars 1.58k forks source link

[Question] Fail to pip install mlc-llm #2974

Open xiboliyaxiangjiaojun opened 1 month ago

xiboliyaxiangjiaojun commented 1 month ago

❓ General Questions

Hi, I’m trying to install mlc-llm on my Jetson agx orin. Environment: Jetson agx orin、Ubuntu 20.04、CUDA 12.2 I ran the following commands: conda create -n mlc python=3.11 conda activate mlc conda install -c conda-forge git-lfs conda install -c conda-forge libgcc-ng and then python -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly-cu122 mlc-ai-nightly-cu122 It seems that I have successfully installed them. Installing collected packages: mlc-llm-nightly-cu122, mlc-ai-nightly-cu122 Successfully installed mlc-ai-nightly-cu122-0.1 mlc-llm-nightly-cu122-0.1 But I failed to import mlc_llm python -c "import mlc_llm; print(mlc_llm)" Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'mlc_llm'

pip list image

Hzfengsy commented 1 month ago

Please try to use a proxy if you are in China mainland :)

xiboliyaxiangjiaojun commented 1 month ago

Please try to use a proxy if you are in China mainland :)

I open the proxy, delete the original virtual environment and tried again, but still failed. image

Hzfengsy commented 1 month ago

Please make sure your proxy works for the domain mlc.ai

There is no tar.gz file in https://mlc.ai/wheels, so it should still be a network problem

xiboliyaxiangjiaojun commented 1 month ago

Please make sure your proxy works for the domain mlc.ai

There is no tar.gz file in https://mlc.ai/wheels, so it should still be a network problem

Maybe I can directly download .whl from https://mlc.ai/wheels, but I can't find one on it for arm64 and linux...

MasterJH5574 commented 1 month ago

Hi @xiboliyaxiangjiaojun, we don't yet have the prebuild wheels for jetson, so as of now you need to build mlc from source. We will try to enable and prebuild wheels, which will take some time though.

SwordFishKe commented 1 month ago

先下载对应的包，再python -m pip install xxx.whl

kinchahoy commented 4 weeks ago

I ran into this problem as well. Building from source isn't working either (can't find tvm.so etc.).

Could you please evaluate the build / install process on arm (RK3588 etc.) boards? There seem to be some recent issues. The original guide at https://blog.mlc.ai/2024/04/20/GPU-Accelerated-LLM-on-Orange-Pi doesn't seem to work anymore.