sourcegraph / cody

AI that knows your entire codebase
https://cody.dev
Apache License 2.0
2.25k stars 215 forks source link

autocomplete: Gemini 1.5 Flash via unstable-gemini provider #4743

Closed abeatrix closed 1 day ago

abeatrix commented 3 days ago

CLOSE https://linear.app/sourcegraph/issue/CODY-2514/implement-autocompletion-provider-for-gemini-models-in-client

This PR is adding Google as the new autocomplete provider using Gemini 1.5 Flash model as experimental/unstable.

This change should be safe as it is hidden behind the "unstable-gemini" setting.

Test plan

  1. Start Cody from this branch
  2. Enable Autocompletes
  3. Update your autocomplete provider settings: "cody.autocomplete.advanced.provider": "unstable-google"
  4. Trigger an autocompletions to verify it works:

image

Examples

image image image image image

abeatrix commented 3 days ago

Thanks, Bee!

I've confirmed that it works locally, which is awesome! However, I encountered the same issues we faced with Claude Instant, which also lacked native infill capabilities. Most of the completions I received don't respect the existing code context. We should dedicate more time to prompt engineering to make it stable.

Based on our past experience, we know this is a challenging problem that we didn't fully resolve with Claude. I suggest we bring this up in Slack and tag leadership and the AI team for their input on whether we should invest more resources into solving this.

Screenshot 2024-07-02 at 13 05 00
Screenshot 2024-07-02 at 13 07 27
Screenshot 2024-07-02 at 13 05 44

Oof I'll play around with the prompt some more tomorrow and we can do a follow up during our 1:1 tmr. Thanks for the through review @valerybugakov !

abeatrix commented 2 days ago

@valerybugakov The updated prompt seems to work 🙏

image image image image image

github-actions[bot] commented 1 day ago

‼️ Hey @sourcegraph/cody-security, please review this PR carefully as it introduces the usage of an unsafe_ function or abuses PromptString.