mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
18.63k stars 1.51k forks source link

[Bug] how to accurately measure the real memory usage on Android ? #2748

Closed MrRace closed 3 days ago

MrRace commented 1 month ago

After deploying Qwen-4B, Qwen-1.8B, and Qwen-0.5B to an Android phone using mlc-llm, I used the following commands to check the system memory usage:

adb shell ps |  findstr "mlc"
adb shell dmabuf_dump PID
adb shell dumpsys meminfo PID

However, the results were all similar, showing only a few hundred MB, so I suspect there might be an issue with the method I'm using. Could you please advise on how to accurately measure the real memory usage when mlc-llm is deployed on a phone, both during model loading and runtime? For example, are there any specific commands you would recommend?

MasterJH5574 commented 1 month ago

Android Studio may have some tools for memory profiling https://developer.android.com/studio/profile/memory-profiler.

Hzfengsy commented 1 month ago

a few hundred MB must be the CPU memory usage. However, the model is stored in the GPU memory, so OS-level memory command is not enough

MrRace commented 1 month ago

a few hundred MB must be the CPU memory usage. However, the model is stored in the GPU memory, so OS-level memory command is not enough

@Hzfengsy Is there a more detailed command?

MrRace commented 1 month ago

Android Studio may have some tools for memory profiling https://developer.android.com/studio/profile/memory-profiler.

@MasterJH5574 You can try, but this method also cannot accurately display memory usage, only a few hundred megabytes.

ponytaill commented 1 month ago

I used Android Studio tools for memory profiling, and got the result. It seems to be correct. The left and right peaks represent two models of different sizes c393468577ea85cd54aaa77057f39dc7

MasterJH5574 commented 3 weeks ago

Yes I could also see accurate memory usage of gigabytes when I used it last time.

pfk-beta commented 2 weeks ago

Almost perfect sollution:

watch -n0.1 'adb shell "cat /proc/meminfo | head"'