Getting error when running locally - The model [undefined] is not found

[chcohen]

this is my docker-compose.yml related code:

  postgres:
    container_name: postgres
    image: "postgres:16.3"
    ports:
      - "5432:5432"
    user: "postgres"
    environment:
      POSTGRES_USER: "postgres"
      POSTGRES_PASSWORD: "root"
      POSTGRES_HOST_AUTH_METHOD: 'scram-sha-256'
      POSTGRES_INITDB_ARGS: '--auth-host=scram-sha-256'
    platform: linux/amd64
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -d postgres" ]
      interval: 30s
      timeout: 10s
      retries: 5

  bedrock:
    container_name: bedrock_connector
    image: "cloudbeer/sample-connector-for-bedrock:0.0.6"
    ports:
      - "8866:8866"
    environment:
      AWS_ACCESS_KEY_ID: 'test-access-key'
      AWS_SECRET_ACCESS_KEY: 'test-secret-key'
      AWS_DEFAULT_REGION: 'us-east-1'
      PGSQL_HOST: '172.17.0.1'
      PGSQL_DATABASE: 'postgres'
      PGSQL_USER: 'postgres'
      PGSQL_PASSWORD: 'root'
      ADMIN_API_KEY: 'my-test-admon-key'
    depends_on:
      - "postgres"

when running this request (it appears in the documentation):

curl -i -X POST 'http://localhost:8866/v1/chat/completions' -H 'Authorization: Bearer br_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' -d '{
  "model": "claude-3-sonnet",
  "messages": [
    {
      "role": "user",
      "content": "ping"
    }
  ],
  "stream": true,
  "temperature": 0,
  "max_tokens": 4096
}'

I'm getting this error: {"success":false,"data":"The model [undefined] is not found. You may refresh to get new models."}

However, when I'm running this request: curl -i -X GET 'http://localhost:8866/v1/models' -H 'Authorization: Bearer br_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

this model id is in the results:

[{"id":"claude-3-sonnet","object":"model","multiple":true,"created":1712297572,"owned_by":"aws-bedrock"},{"id":"claude-3-haiku","object":"model","multiple":true,"created":1712297572,"owned_by":"aws-bedrock"},{"id":"claude-3-opus","object":"model","multiple":true,"created":1712297572,"owned_by":"aws-bedrock"},{"id":"llama3-8b","object":"model","multiple":false,"created":1712297572,"owned_by":"aws-bedrock"},{"id":"llama3-70b","object":"model","multiple":false,"created":1712297572,"owned_by":"aws-bedrock"}]

[cloudbeer]

You may try adding the header: -H "Content-Type: application/json"