Closed chakravarthik27 closed 2 months ago
Resolved: #1127
Env Variables:
API_KEY = [REDACTED] VERSION = "2024-05-13" ENDPOINT = "https://[REDACTED].openai.azure.com/" DEPLOYMENT = 'gpt-4' os.environ['OPENAI_API_KEY'] = API_KEY os.environ['OPENAI_API_VERSION'] = VERSION os.environ['AZURE_OPENAI_ENDPOINT'] = ENDPOINT os.environ['DEPLOYMENT_NAME'] = DEPLOYMENT openai.api_type = "azure" openai.api_base = ENDPOINT openai.api_endpoint = ENDPOINT openai.api_key = API_KEY openai.api_version = VERSION openai.deployment_name = DEPLOYMENT
Harness setup
# Create a Harness object h = Harness( task="question-answering", model={"model": "gpt-4o", 'hub': 'azure-openai', 'max_tokens': 4096}, data={"data_source": "BoolQ", "split": "test-tiny"}, config={ "model_parameters": { "deployment_name": DEPLOYMENT, # get value "temperature": 0.2, "max_tokens": 64 }, "tests": { "defaults": { "min_pass_rate": 1.0 }, "robustness": { "add_typo": { "min_pass_rate": 0.7 }, "lowercase": { "min_pass_rate": 0.7 } } } } ) h.generate().run().report()
Resolved: #1127
Env Variables:
Harness setup