search

spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.21k stars 814 forks source link

temperature and topP for Options should be Double, not Float #935

Open jottinger opened 4 months ago

jottinger commented 4 months ago

Please do a quick search on GitHub issues first, the feature you are about to request might have already been requested.

Expected Behavior

var options=OpenAiChatOptions.builder()
    .withModel("gpt-4o")
    .withTemperature(1.0)
    .build();

Current Behavior

Right now, you have to specify the value as being specifically a float, as javac won't coerce a double to a Float, for use with both temperature and topP. However, in the serialized form being sent to the LLMs, the granularity and format aren't different; the options object is slightly larger with the use of Double instead of Float but that's unlikely to be significant, and it's less confusing for programmers to use the natural syntax for doubles over the syntax for floats. The following code is what one has to do now, if one sets the temperature, for example:

var options=OpenAiChatOptions.builder()
    .withModel("gpt-4o")
    .withTemperature(1.0f)
    .build();

Context

It hasn't really affected me - I mean, I guess I have to type that f or use a Float reference instead, oh the humanity, etc.. but I think it's a relatively appropriate change for quality of life for users of the APIs, and the impact for current users would be minimal since it's a widening operation and not a narrowing one.

ThomasVitale commented 1 month ago

Duplicate of https://github.com/spring-projects/spring-ai/issues/712

ThomasVitale commented 1 week ago

This was solved in https://github.com/spring-projects/spring-ai/issues/712. The issue can be closed.