search

PublicAffairs / openai-gemini

Gemini ➜ OpenAI API proxy. Serverless!
https://my-openai-gemini-demo.vercel.app/
MIT License
337 stars 215 forks source link

The gemini-1.5-flash-002 model does not return data: [DONE] during stream response. #18

Closed zhu327 closed 1 month ago

zhu327 commented 1 month ago

Google has released the gemini-1.5-flash-002 model. I noticed that the stream response is a bit different from the previous gemini-1.5-flash, which causes the current code not to return data: [DONE]. Please fix this. Below is a comparison of the model's returned data.

root@sakura:~# curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:streamGenerateContent?alt=sse&key=xxx" \
        -H 'Content-Type: application/json' \
        --no-buffer \
        -d '{ "contents":[{"parts":[{"text": "Write a story about a magic backpack."}]}]}'
data: {"candidates": [{"content": {"parts": [{"text": "The worn leather"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 3,"totalTokenCount": 11}}

data: {"candidates": [{"content": {"parts": [{"text": " backpack sat in the corner of the dusty attic, a forgotten relic from a time"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 19,"totalTokenCount": 27}}

data: {"candidates": [{"content": {"parts": [{"text": " long past.  Amelia, a girl with wild, untamed hair and eyes that mirrored"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 37,"totalTokenCount": 45}}

data: {"candidates": [{"content": {"parts": [{"text": " the stormy sky, found it tucked behind a chipped chest of drawers.  It wasn't particularly beautiful, with frayed straps and a faded, embroidered emblem of a"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 70,"totalTokenCount": 78}}

data: {"candidates": [{"content": {"parts": [{"text": " swirling, silver moon. But something about it pulled at Amelia, a whisper of adventure calling her name.\n\nThe attic was filled with her grandmother's treasures -"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 104,"totalTokenCount": 112}}

data: {"candidates": [{"content": {"parts": [{"text": " old journals, faded photographs, and a collection of mismatched china. But the backpack held a different kind of magic.  When Amelia slipped it on, a tingle ran down her spine. She reached into the empty pocket, and pulled out a"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "LOW"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 154,"totalTokenCount": 162}}

data: {"candidates": [{"content": {"parts": [{"text": " handful of fresh strawberries, their scent intoxicating.  Moments later, a map materialized, its parchment crackling with age, showing a path leading to a hidden waterfall.\n\nAmelia, consumed by curiosity, followed the map. The journey was fraught with"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 203,"totalTokenCount": 211}}

data: {"candidates": [{"content": {"parts": [{"text": " peril - overgrown paths, treacherous rivers, and a shadowy creature that lurked in the woods. But each time she felt lost or afraid, the backpack came to her aid.  A sturdy rope appeared when she needed to cross a ravine, a warm cloak when the wind whipped around her, and a flashlight when darkness descended.\n\n"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 269,"totalTokenCount": 277}}

data: {"candidates": [{"content": {"parts": [{"text": "The waterfall was a sight to behold, cascading down a sheer cliff, painting the surrounding forest in a mist of rainbows. There, tucked away in a moss-covered cave, she found a lost journal.  It spoke of a forgotten kingdom, hidden deep in the forest, a land of enchantment and wonder. The journal"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 333,"totalTokenCount": 341}}

data: {"candidates": [{"content": {"parts": [{"text": " was a key, revealing a secret code embedded in the moon emblem of the backpack. \n\nAs she deciphered the code, the backpack pulsed with light, opening a portal that shimmered like a mirage.  With a deep breath and a pounding heart, Amelia stepped through. She found herself in a vibrant world, filled with"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 399,"totalTokenCount": 407}}

data: {"candidates": [{"content": {"parts": [{"text": " talking animals, singing trees, and sparkling waterfalls that flowed with liquid sunshine.  \n\nThe kingdom was in peril, threatened by a shadow creature who had stolen its sunlight. Amelia, armed with the magic of her backpack, became its champion. She used its powers to mend fractured friendships, heal wounded spirits, and finally,"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 463,"totalTokenCount": 471}}

data: {"candidates": [{"content": {"parts": [{"text": " confront the shadow creature.  The battle was fierce, but Amelia's courage and the backpack's magic prevailed.  \n\nWith the sunlight restored, the kingdom celebrated Amelia as their hero. They gifted her with a shimmering, silver moonstone, a token of their gratitude.  Back in the attic, Amelia, now"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 529,"totalTokenCount": 537}}

data: {"candidates": [{"content": {"parts": [{"text": " a girl with a knowing glint in her eye, looked at the backpack.  It held the memories of a magical adventure, a reminder that the extraordinary could be found in the most unexpected places, waiting for someone brave enough to open the door. \n"}],"role": "model"},"finishReason": "STOP","index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 579,"totalTokenCount": 587}}

root@sakura:~# curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-002:streamGenerateContent?alt=sse&key=xxx" \
        -H 'Content-Type: application/json' \
        --no-buffer \
        -d '{ "contents":[{"parts":[{"text": "Write a story about a magic backpack."}]}]}'
data: {"candidates": [{"content": {"parts": [{"text": "El"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": "ara wasn't expecting a magic backpack.  She'd expected, perhaps"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": ", a slightly less battered copy of \"Advanced Herbology for the Discer"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": "ning Witch,\" or maybe a slightly less grumpy cat.  Instead, she found it tucked away in her grandmother's attic – a worn leather backpack, smelling"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " faintly of woodsmoke and cinnamon.  It was unremarkable, except for the single, iridescent feather tucked into the stitching.\n\nThe feather pulsed with a"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " faint inner light when Elara touched it.  The backpack, seemingly sensing her curiosity, unzipped with a soft *thwish*.  Inside, instead of the expected emptiness, was a perfectly ripe, sun-kissed peach.\n\n"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": "Elara took the peach. It tasted like sunshine and summer memories.  She ate it slowly, savoring the sweetness.  Then, she reached back into the backpack.  This time, a small, intricately carved wooden flute appeared"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": ".  She played a tentative note – a clear, bell-like sound that echoed strangely, resonating not just in the air, but seemingly within her very bones.\n\nOver the next few weeks, Elara discovered the backpack's true nature.  It was a bottomless well of surprises, tailored to her"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " needs, her wishes, her unspoken desires.  Need a warm blanket on a chilly evening?  The backpack provided.  Craving a specific book?  It materialized.  Feeling lonely? A playful, fluffy kitten appeared, purring contentedly in her lap.\n\nBut the backpack wasn't just about material"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " things.  Once, feeling overwhelmed by a particularly difficult spell, she reached into the backpack, expecting comfort.  Instead, a small, glowing orb appeared.  It pulsed with a gentle warmth, and as she held it, a sense of calm washed over her.  The spell, previously a confusing jumble,"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " suddenly became clear.\n\nThe backpack’s generosity, however, wasn't without limits.  One day, feeling frustrated with her lack of progress in her potion-making class, Elara wished for a perfect, award-winning potion. The backpack obliged, producing a vial filled with a shimmering, iridescent liquid."}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": "  She eagerly submitted it, only to discover, to her horror, it was a powerful sleep draught, not the restorative elixir she'd intended.\n\nThe incident served as a lesson. The backpack didn't provide solutions, it provided tools.  It helped her discover her own resourcefulness, her creativity, her"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " ingenuity.  It was a catalyst for her growth, not a shortcut to success.\n\nElara learned to use the backpack wisely, always asking herself what she truly needed, not just what she wanted.  The worn leather and the iridescent feather became symbols of her journey – a reminder that true magic wasn't about"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": " effortless acquisition, but about the transformative power of self-discovery, fueled by a little bit of wonder and a whole lot of careful consideration.  The magic was within her all along, and the backpack was merely the key that unlocked it.\n"}],"role": "model"}}],"usageMetadata": {"promptTokenCount": 8,"totalTokenCount": 8}}

data: {"candidates": [{"content": {"parts": [{"text": ""}],"role": "model"},"finishReason": "STOP"}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 626,"totalTokenCount": 634}}
anuxs commented 1 month ago

use gemini-1.5-flash-002 model get : 400 error same issue.

ps: gemini-1.5-flash-exp-0827 & gemini-1.5-flash-8b-exp-0924 is OK !

johnd0e commented 1 month ago

@zhu327 @anuxs

Should be fixed now, please check.

zhu327 commented 1 month ago

@johnd0e Good job! Thanks!