stop_sequence terminates if shown in model response

[piotrekwisniewski]

Description of the bug:

Hi everyone,

is this normal that response is terminating while defined stop_sequence shows in response?

I followed gemini API guide an set generation config as proposed (only increased max_output_tokens):

model = genai.GenerativeModel('gemini-1.5-flash') response = model.generate_content( 'Tell me a story about a magic backpack.', generation_config=genai.types.GenerationConfig(

Only one candidate for now.

    candidate_count=1,
    stop_sequences=['x'],
    max_output_tokens=2000,
    temperature=1.0)

)

At gemini API docs there is:

"stopSequences specifies the set of character sequences (up to 5) that will stop output generation. If specified, the API will stop at the first appearance of a stop_sequence. The stop sequence won't be included as part of the response."

Actual vs expected behavior:

Actual behavior:

In my case responding drops out on first "x" shown in the response.

Expected behavior:

Intuitively I thought that responding will be terminated when I pass stop_sequence (in this case "x") from keyboard. And I thought that the same is said on docs by phrase "The stop sequence won't be included as part of the response".

Any other information you'd like to share?

No response

[Gunand3043]

@piotrulawisniewski

The model response will be terminated when it tries to generate any of the words/characters you define in the stop sequence list.

Note: You can't enter more than five words/characters in the stop_sequences list, otherwise, you will encounter an error. For example: stop_sequences=['help', 'the', 'am', 'how', 'ok'].

[github-actions[bot]]

Marking this issue as stale since it has been open for 14 days with no activity. This issue will be closed if no further activity occurs.