search

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
MIT License
1.37k stars 90 forks source link

AssertionError: rwkv_init_from_file failed, check stderr #91

Closed sajjadriaj closed 1 year ago

sajjadriaj commented 1 year ago

Hi,

I am getting this error when trying to load the model. Can someone please help me? I downloaded the quantized model from huggingface 

File [e:\minions\RWKV\rwkv.cpp\rwkv\rwkv_cpp_shared_library.py:90](file:///E:/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py:90), in RWKVSharedLibrary.rwkv_init_from_file(self, model_file_path, thread_count)
     [74](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=73) """
     [75](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=74) Loads the model from a file and prepares it for inference.
     [76](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=75) Throws an exception in case of any error. Error messages would be printed to stderr.
   (...)
     [85](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=84)     Count of layers to load on gpu, must be positive only enabled with cuBLAS.
     [86](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=85) """
     [88](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=87) ptr = self.library.rwkv_init_from_file(model_file_path.encode('utf-8'),
     [89](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=88)                                        ctypes.c_uint32(thread_count))
---> [90](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=89) assert ptr is not None, 'rwkv_init_from_file failed, check stderr'
     [91](file:///e%3A/minions/RWKV/rwkv.cpp/rwkv/rwkv_cpp_shared_library.py?line=90) return RWKVContext(ptr)

AssertionError: rwkv_init_from_file failed, check stderr
saharNooby commented 1 year ago

Hi! There is not enough information in the provided log. rwkv.cpp would write detailed error message just above the Python error, you need to provide it.

But if I would guess by

I downloaded the quantized model from huggingface

probably, you've downloaded the model in an old format, which is not supported by newer version of rwkv.cpp. But, to be sure, I need the specific error message.

sajjadriaj commented 1 year ago

The problem was with the model :) I quantized it myself and it is working fine! Thank you so much for your awesome work!