error loading model: failed to open models/llama/Drararara_llama-13B-ggml: Permission denied

[madhack2142]

models that tried to load, same error log when_trying_to_request.txt

[DielynLandel]

Same issue here...

[hc20k]

Try to put the .bin files directly into the models/llama directory, the script doesn't support walking through folders yet

[DielynLandel]

Try to put the .bin files directly into the models/llama directory, the script doesn't support walking through folders yet

Hello! Thanks for respond!

We tried several models such as ggml-model-q4_0.bin by Drararara.

After a little more than a minute a simple request "Hi!" gives out nonsense:

We only checked the text chat. The bot is silent in the voice chat. All of requirements was installed correctly.

System:

• AMD Ryzen 9 5950X
• 32GB RAM
• Nvidia RTX3090 (24GB)

[hc20k]

DielynLandel

This may be due to the temperature / frequency penalty / presence penalty in the config, I find that a higher frequency penalty (1.1) is better for LLaMA models. Personally I use these settings for LLaMA:

• temperature = 0.8
• presence_penalty = 0.4
• max_tokens = 0
• frequency_penalty = 1.1

I've tested it with pygmalion-7b, wizardlm-7b/13b and this works pretty well.

It also helps LLaMA if you provide a short chat example in your initial prompt, like so:

...your initial prompt

{user_name}: Hello!
{bot_name}: Hi, how's it going today?
{user_name}: Fine, how about you?
{bot_name}: I'm doing well too.

Please let me know if this works out for you, if not I will be happy to help you out some more.