JohnSnowLabs / langtest

Deliver safe & effective language models
http://langtest.org/
Apache License 2.0
506 stars 40 forks source link

Fix/basic setup within datrabricks using azure openai #1128

Closed chakravarthik27 closed 2 months ago

chakravarthik27 commented 2 months ago

Resolved: #1127

Env Variables:

  API_KEY = [REDACTED]
  VERSION =  "2024-05-13"
  ENDPOINT = "https://[REDACTED].openai.azure.com/"
  DEPLOYMENT = 'gpt-4' 

  os.environ['OPENAI_API_KEY'] = API_KEY
  os.environ['OPENAI_API_VERSION'] = VERSION
  os.environ['AZURE_OPENAI_ENDPOINT'] = ENDPOINT
  os.environ['DEPLOYMENT_NAME'] = DEPLOYMENT

  openai.api_type = "azure"
  openai.api_base = ENDPOINT
  openai.api_endpoint = ENDPOINT
  openai.api_key = API_KEY
  openai.api_version = VERSION
  openai.deployment_name = DEPLOYMENT

Harness setup

# Create a Harness object
h = Harness(
    task="question-answering",
    model={"model": "gpt-4o", 'hub': 'azure-openai', 'max_tokens': 4096},
    data={"data_source": "BoolQ", "split": "test-tiny"},
    config={
        "model_parameters": {
            "deployment_name": DEPLOYMENT,  # get value
            "temperature": 0.2,
            "max_tokens": 64
        },
        "tests": {
            "defaults": {
                "min_pass_rate": 1.0
            },
            "robustness": {
                "add_typo": {
                    "min_pass_rate": 0.7
                },
                "lowercase": {
                    "min_pass_rate": 0.7
                }
            }
        }
    }
)
h.generate().run().report()