Doesn't seem to work for me.

[fpena06]

I'm getting the following output when trying to run:

main: seed = 1685215323
starcoder_model_load: loading model from 'models/bigcode/gpt_bigcode-santacoder-ggml-q4_1.bin'
starcoder_model_load: n_vocab = 49280
starcoder_model_load: n_ctx   = 2048
starcoder_model_load: n_embd  = 2048
starcoder_model_load: n_head  = 16
starcoder_model_load: n_layer = 24
starcoder_model_load: ftype   = 1003
starcoder_model_load: qntvr   = 1
starcoder_model_load: ggml ctx size = 1794.97 MB
starcoder_model_load: memory size =   768.00 MB, n_mem = 49152
starcoder_model_load: model size  =  1026.83 MB
main: prompt: 'def fibonnaci('
main: number of tokens in prompt = 7, first 8 tokens: 563 24240 78 2658 64 2819 7

def fibonnaci(!

main: mem per token =   314360 bytes
main:     load time =   344.99 ms
main:   sample time =     0.19 ms
main:  predict time =    62.08 ms / 8.87 ms per token
main:    total time =   446.06 ms

[endo5501]

What happens if "--top_k" argument is non-zero or deleted? If top_k is 0, I think the gpt_sample_top_k_top_p function in common.cpp is accessing out of range of logits_id.

[fpena06]

What happens if "--top_k" argument is non-zero or deleted? If top_k is 0, I think the gpt_sample_top_k_top_p function in common.cpp is accessing out of range of logits_id.

Thank you. That worked.