search

bosun-ai / swiftide

Fast, streaming indexing and query library for AI (RAG) applications, written in Rust
https://swiftide.rs
MIT License
256 stars 18 forks source link

Anthropic Claude support #86

Open timonv opened 4 months ago

timonv commented 4 months ago

Sonnet 3.5 has stellar reviews. If we can get it to SimplePrompt and potentially Embed, it should just work (tm). The Rust ecosystem lacks a mature client, which makes it a bit more work to implement.

fsiefken commented 3 months ago

One could use an existing crate/code for generic LLM interfacing, see for example: https://crates.io/crates/genai https://crates.io/crates/llmclient https://crates.io/crates/allms

According to Anthropic, they don't provide embeddings and refer to VoyageAI Checking the MTEB leaderboard, Stella-EN-1.5B has an unbelievable score compared to VoyageAI, Cohere or ElasticSearch elser-v2: https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/dunzhang/stella_en_1.5B_v5

timonv commented 3 months ago

My concern with existing libraries for Anthropic is that they feel a bit immature, lacking api coverage and important features like automatic backoff, error handling, etc (all of this is doable with reqwest). In comparison, async_openai has all of that and more, and is generic over the configuration, so adding groq support was trivial.

bge-small is the default currently when using fastembed, and has large available as well. It's generally not that hard to add new models to the library unless they require very specific math / operations that need to be translated to Rust (ndarrray is great though).

That said if you feel a library is mature enough, or want to implement it, feel free to suggest it and we can take a look together.