ImportError: DLL load failed while importing KTransformersOps: The specified module was not found.

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Apache License 2.0

745 stars 39 forks source link

ImportError: DLL load failed while importing KTransformersOps: The specified module was not found. #94

Open SCP12rs opened 2 months ago

SCP12rs commented 2 months ago

Error during model run

SCP12rs commented 2 months ago

Everything seems to be in place, but the error won't go away.

I use ktransformers-0.1.1+cu125torch24avx2-cp311-cp311-win_amd64.whl

SCP12rs commented 2 months ago

Tried to build the project via .bat An error occurred.

UnicornChan commented 2 months ago

Do you have CUDA installed and set in the environment variables? I see the final compilation error is due to failing to get the CUDA version.

SCP12rs commented 2 months ago

Maybe I didn't patch it correctly?

SCP12rs commented 2 months ago

Update

UnicornChan commented 2 months ago

I'm very sorry about it. The Windows server I previously used for testing was tested with CUDA 12.5, but not with CUDA 12.6. The script failure might be due to the version update. I will upgrade my Windows server and re-test to verify this issue.

SCP12rs commented 2 months ago

Thank you. I will keep an eye on your project. Very high hopes.