yomorun / yomo

🦖 Stateful Serverless Framework for Geo-distributed Edge AI Infra
https://yomo.run
Apache License 2.0
1.64k stars 129 forks source link

feat(llm): add Cloudflare OpenAI Gateway support #764

Closed fanweixiao closed 3 months ago

fanweixiao commented 3 months ago
name: ai-zipper
host: 0.0.0.0
port: 9000

auth:
  type: token
  token: <YOMO_TOKEN>

bridge:
  ai:
    server:
      addr: localhost:8000
      provider: cloudflare_openai

    providers:
      cloudflare_openai:
        endpoint: https://gateway.ai.cloudflare.com/v1/<ID>/<Endpoint_Name>
        api_key: sk-<OpenAI_API_KEY>
        model: gpt-4-1106-preview
vercel[bot] commented 3 months ago

@fanweixiao is attempting to deploy a commit to the YoMo Team on Vercel.

A member of the Team first needs to authorize it.

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 64.94%. Comparing base (7f08807) to head (79ee097).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #764 +/- ## ========================================== + Coverage 64.48% 64.94% +0.45% ========================================== Files 43 44 +1 Lines 2506 2533 +27 ========================================== + Hits 1616 1645 +29 + Misses 776 775 -1 + Partials 114 113 -1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.