search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.08k stars 1.56k forks source link

[Question] Fail to pip install mlc-llm #2974

Open xiboliyaxiangjiaojun opened 2 weeks ago

xiboliyaxiangjiaojun commented 2 weeks ago

❓ General Questions

Hi, I’m trying to install mlc-llm on my Jetson agx orin. Environment: Jetson agx orin、Ubuntu 20.04、CUDA 12.2 I ran the following commands: conda create -n mlc python=3.11 conda activate mlc conda install -c conda-forge git-lfs conda install -c conda-forge libgcc-ng and then python -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly-cu122 mlc-ai-nightly-cu122 It seems that I have successfully installed them. Installing collected packages: mlc-llm-nightly-cu122, mlc-ai-nightly-cu122 Successfully installed mlc-ai-nightly-cu122-0.1 mlc-llm-nightly-cu122-0.1 But I failed to import mlc_llm python -c "import mlc_llm; print(mlc_llm)" Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'mlc_llm'

pip list image

Hzfengsy commented 2 weeks ago

Please try to use a proxy if you are in China mainland :)

xiboliyaxiangjiaojun commented 2 weeks ago

Please try to use a proxy if you are in China mainland :)

I open the proxy, delete the original virtual environment and tried again, but still failed. image

Hzfengsy commented 2 weeks ago

Please make sure your proxy works for the domain mlc.ai

There is no tar.gz file in https://mlc.ai/wheels, so it should still be a network problem

xiboliyaxiangjiaojun commented 2 weeks ago

Please make sure your proxy works for the domain mlc.ai

There is no tar.gz file in https://mlc.ai/wheels, so it should still be a network problem

Maybe I can directly download .whl from https://mlc.ai/wheels, but I can't find one on it for arm64 and linux...

MasterJH5574 commented 2 weeks ago

Hi @xiboliyaxiangjiaojun, we don't yet have the prebuild wheels for jetson, so as of now you need to build mlc from source. We will try to enable and prebuild wheels, which will take some time though.

SwordFishKe commented 6 days ago

先下载对应的包，再python -m pip install xxx.whl

kinchahoy commented 1 day ago

I ran into this problem as well. Building from source isn't working either (can't find tvm.so etc.).

Could you please evaluate the build / install process on arm (RK3588 etc.) boards? There seem to be some recent issues. The original guide at https://blog.mlc.ai/2024/04/20/GPU-Accelerated-LLM-on-Orange-Pi doesn't seem to work anymore.