search

Codium-ai / pr-agent

🚀CodiumAI PR-Agent: An AI-Powered 🤖 Tool for Automated Pull Request Analysis, Feedback, Suggestions and More! 💻🔍
Apache License 2.0
5.51k stars 510 forks source link

[Feature Request] Support OpenAI seed parameter #1057

Closed MarkRx closed 1 month ago

MarkRx commented 1 month ago

The seed parameter can be used to make the LLM more deterministic on newer OpenAI API versions. See blog for more details.

I propose either

  1. A hard coded seed value is passed in to every request and then logged (when the API supports it) or
  2. A random seed value is passed in to every request and then logged (when the API supports it)

Having access to the seed makes reproducing results easier which in-turn makes debugging and tuning queries easier.

mrT23 commented 1 month ago

Thats a good idea. i will look into that (it requires also making the temperature adaptable)

mrT23 commented 1 month ago

It seems seed is not a working feature yet:

https://community.openai.com/t/seed-param-and-reproducible-output-do-not-work/487245/1

but at least pr-agent will have an infrastructure for that https://github.com/Codium-ai/pr-agent/pull/1063