Open anmolagarwal999 opened 1 year ago
Have you tried more samples?
Have you tried more samples?
I did try this for different prompts (including non-coding based instructions).
I think you have done some wrong, but I cannot figure it out from your config. We check the generated results on humaneval with n=20. They are not the same.
I was trying inference on the HumanEval dataset using
WizardCoder/src/humaneval_gen.py
with the following parameters (state of "generation_config"):All 20 generations for the prompt seem to be exactly the same. I have tried setting a very high temperature (around 10), a high top_p rate (0.99) and the observation still persists. Am I doing something wrong or are the model outputs highly deterministic ?