search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.09k stars 1.57k forks source link

[Bug] Error on running mlc_llm package (Unknown CMake command "tvm_file_glob") #2326

Closed NSTiwari closed 5 months ago

NSTiwari commented 5 months ago

🐛 Bug

I've successfully compiled the Llama-3-8B-Instruct model using q4f16_1 quantization and converted it into Android compatible file (Llama-3-8B-Instruct-q4f16_1-android.tar).

Now, I'm following the official documentation for on-device deployment of Llama-3-8B-q4f16_1 model on Android.

So far, I've correctly followed the prerequisite step i.e., setting the environment variables for ANDROID_NDK, TVM_NDK_CC, TVM_HOME, JAVA_HOME. and installed TVM Unity Compiler as well as MLC LLM Python package using the below command.

python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-ai-nightly python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly mlc-ai-nightly

Now, I'm trying to build the Android app from source. but in Step 2: Build Runtime and Model Libraries, I get the following error: image

Unknown CMake command tvm_file_glob.

To Reproduce

Steps to reproduce the behavior:

  1. cd /home/tiwarinitin1999/mlc-llm/android/MLCChat/
  2. mlc_llm package

Expected behavior

After running mlc_llm package command, I'm expecting the Llama-3-8B-Instruct-q4f16_1-android.tar file in mlc-package-config.json to be successfully produce the libtvm4j_runtime_packed.so and tvm4j_core.jar files in the dist/lib/mlc4j/output folder.

Here is the mlc-package-config.json file. mlc-package-config.json

Please help with this.

Environment

Additional context

tqchen commented 5 months ago

Pleade check the TVM HOME env variable, which should point to 3rd party/tvm without the include path

NSTiwari commented 5 months ago

Thanks, @tqchen. That resolved the issue.

tqchen commented 5 months ago

glad it works

Ammar-Ishfaq commented 1 month ago

I am getting this same bug on my Macbook (Intel, 15.0) After even updating the path from a whole path to a specific path 3rd party/tvm

But still the same issue.

could you please guide me on it? @tqchen @NSTiwari

NSTiwari commented 1 month ago

@Ammar-Ishfaq I have written a detailed blog about the implementation. Try following it.

https://medium.com/google-developer-experts/ml-story-mobilellama3-run-llama3-locally-on-mobile-36182fed3889

Ammar-Ishfaq commented 1 month ago

It looks like you're following the CUDA/LINUX. I'm on MacBook and the solution of updating that path is not working for me.

Could you point out what I'm doing wrong?

-- VERSION: 0.2.00 CMake Error at /Users/muhammad.ammar/Desktop/Projects/mlc-llm/CMakeLists.txt:72 (tvm_file_glob): Unknown CMake command "tvm_file_glob".

NSTiwari commented 1 month ago

That shouldn't make any difference. What's the exact path that you've set?

Ammar-Ishfaq commented 1 month ago

@NSTiwari Here's the path I'm using

Previously:/Users/muhammad.ammar/Desktop/Projects/mlc-llm/3rdparty/tvm

Currently: 3rdparty/tvm

NSTiwari commented 1 month ago

The previous path is still correct.

What's not expected: There's another tvm sub-folder inside the include folder, therefore TVM_HOME shouldn't point to it.

Looking at the path (previous path) you've mentioned, it's correct and it should ideally work.

Ammar-Ishfaq commented 1 month ago

Successfully resolved and compiled.

Steps I followed:

Set TVM_SOURCE_DIR to:

/Users/muhammad.ammar/Desktop/Projects/mlc-llm/3rdparty/tvm

Removed the TVM_HOME variable:

TVM_HOME=/Users/muhammad.ammar/Desktop/Projects/mlc-llm/3rdparty/tvm

Thank you!

NSTiwari commented 1 month ago

Great. A lot of stuff has changed including the environmental variables over the past few months.

Ammar-Ishfaq commented 1 month ago

Absolutely! Despite the changes and challenges with the environment variables, it's rewarding to work through these kinds of issues.