Add stop->stopSequences to OpenAI-Google params mapping

[pchr8]

Hi!

I don't know typescript and I just found out about your project (and I wasn't able to run the proxy locally to test the changes...), but.

When I tried to run an evaluation w/ the EleutherAI eval harness through your public proxy (thank you, BTW!), I got this:

openai.BadRequestError: Error code: 400 - {'error': {'code': 400, 'message': 'Invalid JSON payload received. Unknown name "stop" at \'generation_config\': Cannot find field.', 'status': 'INVALID_ARGUMENT', 'details': [{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'field': 'generation_config', 'description': 'Invalid JSON payload received. Unknown name "stop" at \'generation_config\': Cannot find field.'}]}]}}

I assume it's because the 'when to stop generating' strings in OpenAI are passed through the stop parameter are actually StopSequences in Gemini API: https://ai.google.dev/api/rest/v1/GenerationConfig.

If I understood the code right, this should fix this, but again — I didn't test it.

Thank you for the public proxy again :) Will be a lifesaver as soon as I figure out how to run my evaluations through it

Serhii

[vercel[bot]]

@pchr8 is attempting to deploy a commit to the BrainTrustData Team on Vercel.

A member of the Team first needs to authorize it.

[ankrgyl]

Thank you!! Is there any way you could send a repro command (ideally via cURL) that reproduces the error you're seeing? We can validate the fix before landing it.

[pchr8]

Sure!

# OpenAI - Works
curl -i https://braintrustproxy.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "Write a story about a magic backpack."
      }
    ],
    "seed": 1,
    "stop": ["backpack"]
  }' \
  -H "Authorization: Bearer $OPENAI_API_KEY"

# Gemini - Fails because stop
curl -i https://braintrustproxy.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-pro",
    "messages": [
      {
        "role": "user",
        "content": "Write a story about a magic backpack."
      }
    ],
    "seed": 1,
    "stop": ["backpack"]
  }' \
  -H "Authorization: Bearer $GOOGLE_API_KEY"

# Gemini - Works
curl -i https://braintrustproxy.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-pro",
    "messages": [
      {
        "role": "user",
        "content": "Write a story about a magic backpack."
      }
    ],
    "seed": 1, 
    "stopSequences": ["backpack"]
  }' \
  -H "Authorization: Bearer $GOOGLE_API_KEY"

The specific error for stop w/ Gemini is

{
  "error": {
    "code": 400,
    "message": "Invalid JSON payload received. Unknown name \"stop\" at 'generation_config': Cannot find field.",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "field": "generation_config",
            "description": "Invalid JSON payload received. Unknown name \"stop\" at 'generation_config': Cannot find field."
          }
        ]
      }
    ]
  }
}

[ankrgyl]

@pchr8 this change is now live. Thank you again for the contribution!

[pchr8]

Thank you for making the proxy so easily accessible and for making it open source!