RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
11.99k
stars
825
forks
source link
Got ImportError when using load() to load wkv_cuda #204
I ran this code below and:
wkv_cuda = load(name="wkv", sources=["cuda/wkv_op.cpp", "cuda/wkv_cuda.cu"],
verbose=True,
extra_cuda_cflags=['-res-usage', '--maxrregcount 60', '--use_fast_math', '-O3', '-Xptxas -O3',
f'-DTmax={T_MAX}'])
got this:
Traceback (most recent call last):
File "d:\GitHub\S_GPT\src\model.py", line 56, in
wkv_cuda = load(name="wkv", sources=["cuda/wkv_op.cpp", "cuda/wkv_cuda.cu"],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 1308, in load
return _jit_compile(
^^^^^^^^^^^^^
File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 1736, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 2136, in _import_module_from_library
module = importlib.util.module_from_spec(spec)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 573, in module_from_spec
File "", line 1233, in create_module
File "", line 241, in _call_with_frames_removed
ImportError: DLL load failed while importing wkv: 找不到指定的模块。
What's the wrong with it? Cause it’s my first time dealing with CUDA, I don’t understand it very well. Even after installing VS2022, it’s still the same.
I ran this code below and: wkv_cuda = load(name="wkv", sources=["cuda/wkv_op.cpp", "cuda/wkv_cuda.cu"], verbose=True, extra_cuda_cflags=['-res-usage', '--maxrregcount 60', '--use_fast_math', '-O3', '-Xptxas -O3', f'-DTmax={T_MAX}'])
got this: Traceback (most recent call last): File "d:\GitHub\S_GPT\src\model.py", line 56, in
wkv_cuda = load(name="wkv", sources=["cuda/wkv_op.cpp", "cuda/wkv_cuda.cu"],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 1308, in load
return _jit_compile(
^^^^^^^^^^^^^
File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 1736, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\DELL\anaconda3\Lib\site-packages\torch\utils\cpp_extension.py", line 2136, in _import_module_from_library
module = importlib.util.module_from_spec(spec)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 573, in module_from_spec
File "", line 1233, in create_module
File "", line 241, in _call_with_frames_removed
ImportError: DLL load failed while importing wkv: 找不到指定的模块。
What's the wrong with it? Cause it’s my first time dealing with CUDA, I don’t understand it very well. Even after installing VS2022, it’s still the same.