mvording
commented
8 months ago I was able to get results for this as well the starcoder, starcoderbase models by tweaking the parameter values from the ones provided in the readme.
% ./main --help .... gives some information on parameter flags
below param values returned source code for the function % ./main -m models/bigcode/gpt_bigcode-santacoder-ggml-q4_1.bin -p "def fibonnaci(" --top_k 10 --top_p 0.90 --temp 0.4 % ./main -m models/bigcode/starcoderbase-ggml-q4_1.bin -p "def fibonnaci(" --top_k 10 --top_p 0.90 --temp 0.4
After installing the code and models successfully I ran per the directions in the README. However the output is useless from a code development perspective and does not at all match the README files reported output. Specifically:
./main -m models/bigcode/gpt_bigcode-santacoder-ggml.bin -p "def fibonnaci(" --top_k 0 --top_p 0.95 --temp 0.2 main: seed = 1693067463 starcoder_model_load: loading model from 'models/bigcode/gpt_bigcode-santacoder-ggml.bin' starcoder_model_load: n_vocab = 49280 starcoder_model_load: n_ctx = 2048 starcoder_model_load: n_embd = 2048 starcoder_model_load: n_head = 16 starcoder_model_load: n_layer = 24 starcoder_model_load: ftype = 1 starcoder_model_load: qntvr = 0 starcoder_model_load: ggml ctx size = 3475.60 MB starcoder_model_load: memory size = 768.00 MB, n_mem = 49152 starcoder_model_load: model size = 2707.45 MB main: prompt: 'def fibonnaci(' main: number of tokens in prompt = 7, first 8 tokens: 563 24240 78 2658 64 2819 7
def fibonnaci(!
main: mem per token = 320504 bytes main: load time = 777.92 ms main: sample time = 0.18 ms main: predict time = 84.90 ms / 12.13 ms per token main: total time = 924.55 ms