search

tracehubpm / reports-check-action

Bug Reports (GitHub Issues) Quality Checker
MIT License
10 stars 1 forks source link

Deep Infra API suppport #27

Closed h1alexbel closed 4 months ago

h1alexbel commented 4 months ago

Let's support deep infra models, like Phind/Phind-CodeLlama-34B-v2, we can use it via Deep Infra API

We should define another input called deepinfra_token, and if this input is present, ask the Deep Infra chatbot, but if it does not, then we trying with openai_token

h1alexbel commented 4 months ago

since deep infra aggregates different models, we should have deepinfra_endpoint as well. For Phind-CodeLlama-34B-v2 it should be https://api.deepinfra.com/v1/inference/Phind/Phind-CodeLlama-34B-v2, but we can try to use Open AI compatible API, its here, using this endpoint will be the same for all models ChatGPT like: https://api.deepinfra.com/v1/openai/chat/completions

h1alexbel commented 4 months ago

using Open AI endpoint we can pass model as we are doing now for openai_model input, for Phind-CodeLlama-34B-v2 it will be "model": "Phind/Phind-CodeLlama-34B-v2",

curl "https://api.deepinfra.com/v1/openai/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $(deepctl auth token)" \
  -d '{
      "model": "Phind/Phind-CodeLlama-34B-v2",
      "messages": [
        {
          "role": "user",
          "content": "Hello!"
        }
      ]
    }'