search

salesforce / summary-of-a-haystack

Codebase accompanying the Summary of a Haystack paper.
https://arxiv.org/abs/2407.01370
Apache License 2.0
65 stars 5 forks source link

Configurations of the models #2

Closed PastelBelem8 closed 1 month ago

PastelBelem8 commented 2 months ago

Hello,

Thank you so much for this work! I was wondering if the authors could share what hyperparameters were used to evaluate and generate the summaries. Did you simply use the default OpenAI Chat completion hyperparameters (temperature = 1, top_p = 1) or were other parameters used?

tingofurro commented 2 months ago

Hey @PastelBelem8, good question.

We used default parameters for all models (temperature = 1, top_p = 1). It would be interesting to see what effect the generation parameters would have on the results, but we have not explored this yet! Happy to hear your thoughts,

Philippe