[Security solution] Fix gemini streaming

stephmilovic commented 3 days ago

Summary

Fixes streaming for Gemini in Security Assistant.

Content can appear in the finishReason block. I'm not sure when this started happening. Updates our streaming logic to support content being in the finishReason block.

Example of finishReason block with content:

`data: {"candidates": [{"content": {"role": "model","parts": [{"text": " are 170 critical and 20 high open alerts."}]},"finishReason": "STOP","safetyRatings": [{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE","probabilityScore": 0.060086742,"severity": "HARM_SEVERITY_NEGLIGIBLE","severityScore": 0.17106095},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE","probabilityScore": 0.16776322,"severity": "HARM_SEVERITY_LOW","severityScore": 0.37113687},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE","probabilityScore": 0.124212936,"severity": "HARM_SEVERITY_NEGLIGIBLE","severityScore": 0.17441037},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE","probabilityScore": 0.05419875,"severity": "HARM_SEVERITY_NEGLIGIBLE","severityScore": 0.03461887}]}],"usageMetadata": {"promptTokenCount": 1062,"candidatesTokenCount": 15,"totalTokenCount": 1077},"modelVersion": "gemini-1.5-pro-002"}`

To test

Have alerts
Have a Gemini connector (gemini-1.5-pro-002)
Have streaming on in the assistant with the Gemini connector selected
Ask the assistant: "How many open alerts do I have?"

Previously

A response begin to streams and then the response gets cut off.

Now

The response streams in full as expected.

elasticmachine commented 3 days ago

Pinging @elastic/security-solution (Team: SecuritySolution)

stephmilovic commented 3 days ago

@elasticmachine merge upstream

elasticmachine commented 3 days ago

:yellow_heart: Build succeeded, but was flaky

Buildkite Build
Commit: 8641e85de3217557c08a533a1cbe80e7751610d7

Failed CI Steps

Test Failures

[job] [logs] Jest Tests #4 / form payload & api errors should surface the API errors from the put HTTP request
[job] [logs] FTR Configs #98 / management index pattern not found redirects to the main view if data view is missing