BlinkDL / RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
11.99k stars 825 forks source link

Got ImportError when using load() to load wkv_cuda #204

Open nanjunye opened 7 months ago

nanjunye commented 7 months ago

I ran this code below and: wkv_cuda = load(name="wkv", sources=["cuda/wkv_op.cpp", "cuda/wkv_cuda.cu"], verbose=True, extra_cuda_cflags=['-res-usage', '--maxrregcount 60', '--use_fast_math', '-O3', '-Xptxas -O3', f'-DTmax={T_MAX}'])

got this: Traceback (most recent call last): File "d:\GitHub\S_GPT\src\model.py", line 56, in wkv_cuda = load(name="wkv", sources=["cuda/wkv_op.cpp", "cuda/wkv_cuda.cu"], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 1308, in load return _jit_compile( ^^^^^^^^^^^^^ File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 1736, in _jit_compile return _import_module_from_library(name, build_directory, is_python_module) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 2136, in _import_module_from_library module = importlib.util.module_from_spec(spec) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 573, in module_from_spec File "", line 1233, in create_module File "", line 241, in _call_with_frames_removed ImportError: DLL load failed while importing wkv: 找不到指定的模块。

What's the wrong with it? Cause it’s my first time dealing with CUDA, I don’t understand it very well. Even after installing VS2022, it’s still the same.

BlinkDL commented 3 months ago

run it in vs2022 "x64 native tools command prompt"

Lixuanhe commented 1 month ago

I removed "-Xptxas -O3" from wkv6_cuda and that solved the problem.