search

Telosnex / fllama

llama.cpp for Flutter
Other
99 stars 11 forks source link

Bug: error loading model: check_tensor_dims: tensor 'token_embd.weight' not found #9

Open LukeMoody01 opened 3 weeks ago

LukeMoody01 commented 3 weeks ago

Cloned the repo, and spun up the example project on macos, and I get the error:

error loading model: check_tensor_dims: tensor 'token_embd.weight' not found

After selecting a gguf file and clicking "Run" with flashcards schema, and any prompt:

image

This also blocks using the package as well as we cannot load any models due to this error.

LukeMoody01 commented 3 weeks ago

More info (Debug logs after clicking run):

flutter: [llama.cpp] Error: Unable to load model.
fllama_get_eos_token: tokens_idx: 13
fllama_get_eos_token: word: <|endoftext|>
fllama_get_bos_token: tokens_idx: 13
fllama_get_bos_token: word: <|endoftext|>
fllama_get_eos_token: tokens_idx: 13
fllama_get_eos_token: word: <|endoftext|>
fllama_get_bos_token: tokens_idx: 13
fllama_get_bos_token: word: <|endoftext|>
flutter: [fllama] Using built-in chat template: {% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system
You are a helpful assistant<|im_end|>
' }}{% endif %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}
[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: type 'String' is not a subtype of type 'int' of 'index'
#0      formatFunctionDefinition (package:fllama/misc/openai_tool.dart:53:55)
openai_tool.dart:53
#1      Tool.typescriptDefinition (package:fllama/misc/openai_tool.dart:22:12)
openai_tool.dart:22
#2      fllamaApplyChatTemplate.<anonymous closure> (package:fllama/fllama_universal.dart:187:19)
fllama_universal.dart:187
#3      MappedListIterable.elementAt (dart:_internal/iterable.dart:425:31)
iterable.dart:425
#4      ListIterable.join (dart:_internal/iterable.dart:159:25)
iterable.dart:159
#5      fllamaApplyChatTemplate (package:fllama/fllama_universal.dart:188:8)
fllama_universal.dart:188
#6      fllamaChat (package:fllama/fllama_universal.dart:123:10)
fllama_universal.dart:123
<asynchronous suspension>
#7      _MyAppState._runInferencePressed (package:fllama_example/main.dart:512:21)
main.dart:512
<asynchronous suspension>
wajeehulhassanvii commented 1 week ago

@LukeMoody01 Did you find any solution to it?