keirp / automatic_prompt_engineer

MIT License
1.16k stars 149 forks source link

Error when using new model - gpt-3.5-turbo-instruct-0914 #13

Open Mosh0110 opened 1 year ago

Mosh0110 commented 1 year ago

First I encountered the error: "Too many parallel completions requested. You submitted..."

This was fixed after lowering the batch size to 3 in both the the generation and the evaluation. (prompt_gen_batch_size=3, eval_batch_size=3)

After that I'm getting a division by zero error from the lines: 87 prompt_log_probs[-1].append(sum(lps) / len(lps)) 88 i += 1 89 return prompt_log_probs ZeroDivisionError: division by zero

Taken from the function _compute_avg_likelihood in the likelihood file.

What can I do?