GPU usage in Activity Monitor

ggerganov / llama.cpp

LLM inference in C/C++

MIT License

67.91k stars 9.74k forks source link

GPU usage in Activity Monitor #1878

Closed Vadim170 closed 1 year ago

Vadim170 commented 1 year ago

Is it normal that I don't see GPU usage in the Mac OS Activity Monitor when running Metal-based computations?

ianscrivener commented 1 year ago

I have an M2 MacbookPro and see GPU % in the 90s when running llama.cpp

this is my helper script that I source to fetch and build the latest llama.cpp code

conda activate gg.3.9.16
git pull
git status
make clear
LLAMA_METAL=1 make -j

TBH: I haven't had much luck building with CMake...

Vadim170 commented 1 year ago

@ianscrivener I tried to follow your instructions, there were no changes. I also tried to build via cmake:

mkdir build-metal
cd build-metal
cmake -DLLAMA_METAL=ENABLED ..
cmake -build . --configuration release

However, I noticed that only the CPU is being used and there is no GPU-related activity. The repository is exactly the latest version, the project was cleaned before the build.

Vadim170 commented 1 year ago

I apologize for the oversight. It appears that I neglected to include the "-ngl" parameter in the commands.