How to set temprature and output token size in the chat mode?

[jojogh]

Is there any examples to guide how to set temprature and output token size in the chat mode?

[eliranwong]

You may read this example:

examples/fill-in-middle/main.py

[jojogh]

thanks first, i have seen this example, it is generate mode, not in chat mode.

[eliranwong]

For example, I use the following for one of my projects:

from ollama import Options

        completion = ollama.chat(
            #keep_alive=0,
            model=config.ollamaDefaultModel,
            messages=[
                *ongoingMessages,
                {
                    "role": "user",
                    "content": prompt,
                },
            ],
            format="json",
            stream=True,
            options=Options(
                temperature=0.0,
                num_ctx=8192,
                num_predict=-1,
            ),
        )

[deepcoder]

and "keep_alive" as well, thanks!

[jojogh]

For example, I use the following for one of my projects:

from ollama import Options

        completion = ollama.chat(
            #keep_alive=0,
            model=config.ollamaDefaultModel,
            messages=[
                *ongoingMessages,
                {
                    "role": "user",
                    "content": prompt,
                },
            ],
            format="json",
            stream=True,
            options=Options(
                temperature=0.0,
                num_ctx=8192,
                num_predict=-1,
            ),
        )

Thanks Wong, I will try. Many thanks.