search

google / gemma.cpp

lightweight, standalone C++ inference engine for Google's Gemma models.
Apache License 2.0
5.98k stars 508 forks source link

the 7b-pt-sfp doesn‘t looks like normal. some thing wrong with the args? #27

Closed cagev closed 8 months ago

cagev commented 8 months ago

use the build gemma.cpp to run 2b, it's ok . but 7b seems abnoraml. something wrong?

./gemma --tokenizer tokenizer.spm --compressed_weights 7b-pt-sfp.sbs --model 7b-it

/ ` |/ \ ' ` | ' ` \ / _` | / | ' | ' \ | (_| | _/ | | | | | | | | | | (| || (| |) | |) | __, |_|| || ||| || ||_,()| ./| ./ __/ | | | | | |__/ || |_|

tokenizer : tokenizer.spm compressed_weights : 7b-pt-sfp.sbs model : 7b-it weights : [no path specified] max_tokens : 3072 max_generated_tokens : 2048

Usage Enter an instruction and press enter (%Q quits).

Examples

Compute the nth fibonacci number in javascript.

[ Reading prompt ] ................

The Fibonacci sequence is a sequence of numbers in which each number is the sum of the two preceding numbers. The sequence begins: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765, 10946, 17701, 28647, 46348, 75000, 121348, 196348, 317696, 514044, 831740, 1345784, 2177524, 3523308, 5700832, 9224140, 14924972, 24149112, 39074084, 63223196, 102297280, 165520476, 267817756, 433338232, 701199988, 1134538220, 1835738208, 2970976428, 4806714636, 7777690864, 12584405498, 20361896362, 32943361860, 53305258222, 86245619082, 139550876304, 225796495386, 365347371690, 591240867076, 956588244766, 1547829111842, 2498417956608, 4046247068450, 654072465058, 1058727133508, 1712799600568, 2771526734076, 4484326334644, 7255853068720, 11740179403364, 18996032470964, 30736211874328, 49732244345292, 79468456219616, 129200699564912, 208669155784528, 337870855349440,

./gemma --tokenizer tokenizer.spm --compressed_weights 7b-pt-sfp.sbs --model 7b-it

/ ` |/ \ ' ` | ' ` \ / _` | / | ' | ' \ | (_| | _/ | | | | | | | | | | (| || (| |) | |) | __, |_|| || ||| || ||_,()| ./| ./ __/ | | | | | |__/ || |_|

tokenizer : tokenizer.spm compressed_weights : 7b-pt-sfp.sbs model : 7b-it weights : [no path specified] max_tokens : 3072 max_generated_tokens : 2048

Usage Enter an instruction and press enter (%Q quits).

Examples

write a quick sort algorithm.

[ Reading prompt ] ...............

/*

jan-wassenberg commented 8 months ago

In the command line we seem to have a mismatch between PT (pretrained) and IT (instruction tuned). I think you want to download+use the IT model for your use case.

austinvhuang commented 8 months ago

+1 - for following instructions "it" for "instruction tuned" - 7b-it-sfp or 2b-it-sfp. Closing for now but let us know if this doesn't resolve your issue.