Helicone / helicone

🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23
https://www.helicone.ai
Apache License 2.0
1.7k stars 185 forks source link

Anthropic Stream Issue #1666

Open samtamp95 opened 5 months ago

samtamp95 commented 5 months ago

Path: /getting-started/integration-method/anthropic

Is there not away to enable anthropic so that it streams?

I tried using openai's but anthropic isn't really compatible with it.

any suggestions on the best to use anthropic and helicone but stream the text?

colegottdank commented 5 months ago

Hi, we do support streaming for anthropic. Here is an example curl:

curl --request -N POST \
  --url https://anthropic.hconeai.com/v1/messages \
  --header 'Content-Type: application/json' \
  --header 'Helicone-Auth: Bearer {HELICONE_API_KEY}' \
  --header 'anthropic-version: 2023-06-01' \
  --header 'x-api-key: {ANTHROPIC_API_KEY}' \
  --data '{
    "model": "claude-3-opus-20240229",
    "max_tokens": 100,
    "stream": true,
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Tell me a story"
                }
            ]
        }
    ]
}'
aoganesyan commented 4 days ago

same issue here