Open Maykeye opened 6 months ago
yeah because you need to do os.environ['RWKV_CUDA_ON'] = '1' before import rwkv
yeah because you need to do os.environ['RWKV_CUDA_ON'] = '1' before import rwkv
Yes, exactly! This is exactly is the issue. Normal imports look like this:
(BEGINNING OF FILE)
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS
which is exactly the case with transformers, exllama, llama.cpp, HQQ(which just like RWKV has several backends but only one can be used at the time)
Not like this
import os
os.environ["RWKV_CUDA_ON"] = f"1"
if "I don't want standard formatter to break my code":
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS
Running autopep8 should not break the code. Which it does for no reasonable reason unless something like this if
fence is used.
At the very least it's possible to make couple of QoL files like rwkv/cuda_jit
like
import os
os.environ["RWKV_CUDA_ON"] = "1"
os.environ["RWKV_JIT_ON"] = "1"
if "imports come after os":
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS
so user code can call from rwkv.cuda_jit import RWKV, PIPELINE
without caring if it's environ["RWKV_CUDA_ON"]="1"
or environ["RWKV_CUDA"]="ON"
which is impossible to tell using LSP.
Autopep8 is a friend! It shouldn't be worked around
Expected behavior:
Running something like
autopep8 mycode.py
on the code) that uses RWKV should not break the code. Pressing "ctrl-s" in vs code with "autoformat on save" should not break the code.It does: linter moves imports where they belong, to the very top of the file, above any statements like
os.environ[]=1
.Actual behavior:
from torch.utils.cpp_extension import load is not executed if environ is not set up during import.
Therefore simple
is unable to load the model