openai o1 model setting

[dtrckd]

Hi,

The openai o1 models doesn't seem to support a temperature different than one and the stream mode . Thus, would it be possible to set a per model setting, in order to automatically set a temperature, and stream=false when switching to it (like o1-mini for example) ?

[sigoden]

o1 is the only one among all models that doesn't support streaming. This is not a common situation, and we will not introduce a stream model configuration.

Currently, o1 has many flaws. Besides not supporting stream, it also lacks support for role (system message) and function calling (tool use). It's not recommended for use with aichat.

https://platform.openai.com/docs/guides/reasoning/beta-limitations

[InfImpDrive]

@sigoden this may need a re-evaluation if a lot of models start using async CoT. There's a surgical use case where it would be quite useful in ways superior to other models.

o1 is now available on Tier 4 (250 lifetime spend; can be just a 250 deposit for new users) and did some tests using other tools.

It's absurdly fantastic for coding questions that requires reasoning such as these questions, some of which can be piped to files:

Take a look at [[file1.js]], [[file2.js]] and [[file3.js]]. Analyze similarities in these files and then please generate a new suggested [[baseclass.js]] as a refactored base class that I can use for these files. Output only the baseclass.js code, code only, no commentary.

Or

Take a look at [[test.js]], [[test.html]], [[test.css]]. Please explain why the page is loading blank."

o1 succeeded on these for things that ChatGPT 4o / Claude failed on.

It works interestingly well when I tested this in the Playground, and it produces massively better debugging / ideas / explanations for multi-file tasks than most other models, for these types of reasoning tasks in a way that seems 10x more reliable than non-o1.

It also has a 128K input context window and 64K output, so it can output large modules. You'd use a command line to cat in a few modules (with separators in between like [[[start of file1.js]]] separating the concatenated files into it. And outputs much superior code for certain kinds of requests.

Automating this is hard, and having o1 added would be a big boon. Some of us would pay for some opensource dev time for this addition.

Although it might be too much of a hassle to add non-streaming models right now, this may necessarily change in the future with 2025's CoT boom. Or maybe wait-and-see what streaming workflows are introduced, or that someone else puts a streaming wrapper around o1-mini, with an intermediate model.

It can be useful to use a command line to quickly dump a whole project to aichat o1 and asking a query:

for i in ./framework/*; do echo -e "--- Contents of file $i ---\n"; cat $i; done | aichat "Please explain why I am getting a cryptic javascript error 0x0000 when running processor.js"

And for more complex refactoring tasks that requires more advanced reasoning that is too complicated for other models

cat advancedMathFormula.cpp | aichat "Please rewrite the math formula to use AVX512 matrix math in assembly instructions in an asm block. Output ONLY the code" > alternateMathFormula.cpp

diff advancedMathFormula.cpp alternateMathFormula.cpp

My experience is o1 is superior for complicated CoT assistance over multi-code-file tasks than most other models. The ability to just write quick shell command to surgically spew subsets of a whole code project into aichat works but other LLMs aren't as good for major multifile tasks where specific AI-assisted debugging/refactoring tasks requires CoT behavior rather than simpler rote/autocomplete behaviors.

[sigoden]

Feel free to use o1 with --no-stream/-S or .set stream false.

AIChat does not automatically set stream=false for special models (like openai:o1*), which does not mean it does not support non-streaming.

[InfImpDrive]

Oh! I was unaware of this setting. Thanks.