Thanks for the great work and codes!
While I look through the codes, I notice a possible discrepancy between the paper and the codes.
The paper states that you used beam search with num_beams=5 and do_sample=True to obtain the most likely answers. On the other hand, the generate.py script is using the beam search with do_sample=False instead. Am I missing something here?
Hi Lorenz,
Thanks for the great work and codes! While I look through the codes, I notice a possible discrepancy between the paper and the codes. The paper states that you used beam search with
num_beams=5anddo_sample=Trueto obtain the most likely answers. On the other hand, thegenerate.pyscript is using the beam search withdo_sample=Falseinstead. Am I missing something here?https://github.com/lorenzkuhn/semantic_uncertainty/blob/27adbf0dc1bf056c771c205d89c2a79cbd82dc3a/code/generate.py#L113-L130