search

Atome-FE / llama-node

Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.
https://llama-node.vercel.app/
Apache License 2.0
862 stars 62 forks source link

Unsupported file version 101 when loading rwkv #101

Open linonetwo opened 1 year ago

linonetwo commented 1 year ago

https://github.com/saharNooby/rwkv.cpp/blob/master/docs/FILE_FORMAT.md#101

maybe rwkv.cpp is not updated @yorkzero831 ？

And what is this /Users/runner/work/llama-node/llama-node/packages/rwkv-cpp/rwkv-sys/rwkv.cpp/rwkv.cpp path?

Unsupported file version 101
/Users/runner/work/llama-node/llama-node/packages/rwkv-cpp/rwkv-sys/rwkv.cpp/rwkv.cpp:195: version == RWKV_FILE_VERSION
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                        ' was expected to have ID '50254' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                       ' was expected to have ID '50255' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                      ' was expected to have ID '50256' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                     ' was expected to have ID '50257' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                    ' was expected to have ID '50258' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                   ' was expected to have ID '50259' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                  ' was expected to have ID '50260' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                 ' was expected to have ID '50261' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '                ' was expected to have ID '50262' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '               ' was expected to have ID '50263' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '              ' was expected to have ID '50264' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '             ' was expected to have ID '50265' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '            ' was expected to have ID '50266' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '           ' was expected to have ID '50267' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '          ' was expected to have ID '50268' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '         ' was expected to have ID '50269' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '        ' was expected to have ID '50270' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '       ' was expected to have ID '50271' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '      ' was expected to have ID '50272' but was given ID 'None'
[Tue, 18 Jul 2023 14:34:42 +0000 - WARN - tokenizers::tokenizer::serialization] - Warning: Token '     ' was expected to have ID '50273' but was given ID 'None'
[1]    26853 segmentation fault  node "xxx.js"

then electron is crash and quit, like https://github.com/Atome-FE/llama-node/issues/76#issuecomment-1620304479