search

ggerganov / ggml

Tensor library for machine learning
MIT License
11.22k stars 1.04k forks source link

vulkan: multithread pipeline creation #963

Closed jeffbolznv closed 1 month ago

MaggotHATE commented 2 months ago

Thank you! It works, shortens startup greatly. Though, does it suppose to use CPU and not GPU?

jeffbolznv commented 1 month ago

Thank you! It works, shortens startup greatly. Though, does it suppose to use CPU and not GPU?

All compiles occur on the CPU, this just uses multiple threads to do them in parallel.

jeffbolznv commented 1 month ago

Ping for review.

ggerganov commented 1 month ago

cc @0cc4m

jeffbolznv commented 1 month ago

I do most of my development on Windows, it works well there.

MaggotHATE commented 1 month ago

you tested it on Windows, right?

Yes, I did, although I use w64devkit, if that's important. I don't have MSVS installed to test there.