search

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
MIT License
1.37k stars 90 forks source link

Invalid file header #120

Closed linonetwo closed 1 year ago

linonetwo commented 1 year ago

(venv) PS E:\repo\rwkv.cpp> python E:\repo\rwkv.cpp\rwkv\quantize.py C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096.pth C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096-Q5_1.bin Q5_1 Loading model from 'C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096.pth'

E:\repo\rwkv.cpp\rwkv.cpp:250: header.magic == 0x67676d66 Invalid file header E:\repo\rwkv.cpp\rwkv.cpp:1782: rwkv_fread_file_header(in_file.file, in_header) Traceback (most recent call last): File "E:\repo\rwkv.cpp\rwkv\quantize.py", line 31, in main() File "E:\repo\rwkv.cpp\rwkv\quantize.py", line 22, in main library.rwkv_quantize_model_file( File "E:\repo\rwkv.cpp\rwkv\rwkv_cpp_shared_library.py", line 254, in rwkv_quantize_model_file assert self.library.rwkv_quantize_model_file( AssertionError: rwkv_quantize_model_file failed, check stderr

linonetwo commented 1 year ago

Even raven model have this error, so it is not World model's fault.

python E:\repo\rwkv.cpp\rwkv\quantize.py C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-Raven-14B-v10-Eng99%-Other1%-20230427-ctx8192.pth C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-Raven-14B-v10-Eng99%-Other1%-20230427-ctx8192-Q5_1.bin Q5_1
Loading model from 'C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-Raven-14B-v10-Eng99%-Other1%-20230427-ctx8192.pth'

E:\repo\rwkv.cpp\rwkv.cpp:250: header.magic == 0x67676d66
Invalid file header
E:\repo\rwkv.cpp\rwkv.cpp:1782: rwkv_fread_file_header(in_file.file, in_header)
Traceback (most recent call last):
  File "E:\repo\rwkv.cpp\rwkv\quantize.py", line 31, in <module>
    main()
  File "E:\repo\rwkv.cpp\rwkv\quantize.py", line 22, in main
    library.rwkv_quantize_model_file(
  File "E:\repo\rwkv.cpp\rwkv\rwkv_cpp_shared_library.py", line 254, in rwkv_quantize_model_file
    assert self.library.rwkv_quantize_model_file(
AssertionError: rwkv_quantize_model_file failed, check stderr
linonetwo commented 1 year ago

python rwkv\convert_pytorch_to_ggml.py C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096.pth C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096-FP16.bin FP16

FP16 work without problem ATM

linonetwo commented 1 year ago

python E:\repo\rwkv.cpp\rwkv\quantize.py C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096-FP16.bin C:\Users\linonetwo\Documents\model\LanguageModel\RWKV\RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096-Q5_0.bin Q5_0

This works. quantize.py only accept .bin as input, so need to convert .pth to .bin first, then quantize it.