[Feature Request] Setting for KI-Model: Temperature, Frequency Penalty, Presence penalty, Top P, Cut-off Limit

[hienstorfer]

ChatGPT is providing settings to have more control over the execution of the promt. Maybe it is possible to make these settings available per promt- With these settings, you can control:

1. Temperature: Controls randomness and creativity.
2. Top P (Nucleus Sampling): Limits selection to the most probable tokens, enhancing control over diversity.
3. Frequency Penalty: Reduces repetition of the same tokens.
4. Presence Penalty: Encourages introducing new tokens and concepts.
5. Cut-off Limit: Sets a maximum length for the response.

[piazzatron]

Thanks for writing this up @hienstorfer. It's a good power user feature feature, though even I don't have an intuitive sense of the difference between some of these (for example temperature vs "presence penalty") - would you want all of these as options, or are there some that you think are more valuable than others?

[hienstorfer]

Hello, I prioritized the settings. Most important one is temperature. This is the one I would choose if I had only one choice. I made up examples as well that might make it clearer.

1. Temperature: Controls randomness and creativity. A higher temperature value (e.g., 1.0) makes the output more random, leading to more creative and varied responses. A lower temperature value (e.g., 0.2) makes the output more deterministic, producing more focused and predictable responses. example: High Temperature (1.0): "The sun is shining brightly in the sky, casting golden rays on the bustling streets." Low Temperature (0.2): "The sun is shining in the sky."

2. Frequency Penalty: Reduces repetition of the same tokens. A higher frequency penalty discourages the model from using the same words or phrases multiple times within the same response. Example: High Frequency Penalty: "The cat sat on the mat. It then moved to the rug." Low Frequency Penalty: "The cat sat on the mat. The cat sat on the mat again."

3. Presence Penalty: Encourages introducing new tokens and concepts. This penalty increases the model’s creativity by encouraging it to introduce new topics or concepts instead of sticking to the same ones. It is particularly useful for avoiding repetitive responses and ensuring varied content. example: High Presence Penalty:"The boy played soccer and then read a book." Low Presence Penalty:"The boy played soccer. The boy played soccer."

[hienstorfer]

Prompt: "Tell me about project management."

High Temperature Example (1.0) "Project management is like orchestrating a grand symphony, where each team member is an instrumentalist contributing to a harmonious performance. Imagine a dynamic journey filled with strategic planning, resource juggling, and the thrill of meeting deadlines. It’s a realm where creativity meets organization—crafting innovative solutions, adapting to unexpected challenges, and leading a diverse team toward a shared vision. With tools like Agile and Scrum, you’re not just managing tasks; you’re fostering a culture of collaboration and continuous improvement. Think of project management as the art of turning ideas into reality, one milestone at a time, with a touch of finesse and a dash of innovation."

Low Temperature Example (0.2) "Project management involves planning, organizing, and managing resources to achieve specific goals within a defined timeline. It includes defining project objectives, creating a project plan, assigning tasks, and monitoring progress. Key methodologies used in project management include Agile, Scrum, and Waterfall. Effective project management ensures that projects are completed on time, within budget, and to the required quality standards. It requires skills in communication, time management, and problem-solving. Project managers use tools like Gantt charts, project management software, and performance metrics to keep projects on track and manage risks."

[piazzatron]

Temperature coming in the next release, due in a few weeks 👍. Will have a global default, as well as per-prompt.

[piazzatron]

Hi @hienstorfer, temperature is in the new V2 release that's out today. Can be set per prompt as well.