Open evilsocket opened 1 month ago
the model responds well when prompt template is not used (https://github.com/evilsocket/cake/blob/main/cake-core/src/models/llama3/llama.rs#L266)
this is also happening in candle llama example code -> https://github.com/huggingface/candle/issues/2341
Has this been fixed? I met the same problem.
@caohuilong the issue is open, the bug has not been fixed, waiting for the candle team to respond
the tokenizer has issues resolving a few tokens including special ones (they will be shown in the output as), which is causing all sorts of gibberish output ... it's probably a matter of parsing the model/tokenizer.json properly