add google support for llm-client (aka llm-polyglot)

[jcamera]

llm-polyglot support for Google AI

This is using Google AI SDK (aka Gemini API) for Javascript - https://github.com/google-gemini/generative-ai-js

Be sure to set GOOGLE_API_KEY in your environment for use.

Features added:

• Chat completion with support for message arrays conforming to OpenAI spec
  • google content array "parts" supports "model" (instead of "assistant") and "user", while system messages are may be included in the systemInstruction field
• Support for streaming chat completion
• Function calling - streaming and non-streaming
• Support for Google AI Cache Manager - cache data may be added through createCacheManager and used by passing in the cache name from the response into chat.completions.create by setting it in { additionalProperties: cacheName }} (note: this requires a paid account)

Note: polyfill added due to missing support for TextEncoderStream in bun.js (https://github.com/oven-sh/bun/issues/5648)

we may also consider using their Vertex AI API which has some tentative support for using OpenAI libraries with Gemini - https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library

or directly using their Gemini Rest API - https://ai.google.dev/api/rest

Feedback is welcome :-)

[changeset-bot[bot]]

⚠️ No Changeset found

Latest commit: 7a91550f0ac988f666454a84a0b920789c5df34d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
island-ai-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 16, 2024 5:00pm

[ZECTBynmo]

Cool stuff @jcamera! Have you thought about how this might interact with/support the caching features that Gemini has? It'd be an awesome capability to add.

[jcamera]

Cool stuff @jcamera! Have you thought about how this might interact with/support the caching features that Gemini has? It'd be an awesome capability to add.

Hey thank you @ZECTBynmo! I'm just getting my feet wet here, but I did look at an example using GoogleAICacheManager. We could totally add methods on the Google provider to support this, and use the same input data types as the chat completion. And we could then add a param on chat.completions.create like { cache: name } using the name returned when the cache is set. This would diverge from the OpenAI spec a bit, but shouldn't hurt anything as an extra optional field. If OpenAI eventually adds a cache field we'd need to update. :-) I can start adding this into this PR unless any objections.

[ZECTBynmo]

I can start adding this into this PR unless any objections.

Awesome!

[jcamera]

Hi @ZECTBynmo, fyi I added some pieces to allow using the Google cache manager. (A paid account is needed to use it) Added an example in /examples. I'd be curious to learn about your use cases, and happy to help if any issues! 🙂