search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.36k stars 2.71k forks source link

azure-ai-generative not compatible with latest azureml-metrics #33890

Open pamelafox opened 5 months ago

pamelafox commented 5 months ago

Describe the bug A clear and concise description of what the bug is.

We now get this error when calling evaluate(): from azure.ai.generative.evaluate import evaluate File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/init.py", line 8, in from ._evaluate import evaluate File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 22, in from azure.ai.generative.evaluate._metric_handler import MetricHandler File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/_metric_handler.py", line 6, in from azure.ai.generative.evaluate._constants import TYPE_TO_KWARGS_MAPPING File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/_constants.py", line 13, in QA: constants.QUESTION_ANSWERING, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: module 'azureml.metrics.constants' has no attribute 'QUESTION_ANSWERING'

To Reproduce Steps to reproduce the behavior:

  1. git checkout https://github.com/Azure-Samples/ai-rag-chat-evaluator/ at commit fe5bc695c5864ec0b2d3f2b40ff0744a2055042b
  2. Try to run python3 -m scripts generate --output=example_input/qa.jsonl --numquestions=14 --persource=2

Expected behavior

No error. This worked yesterday so I suspect it was the release 12 hours ago that broke it.

kashifkhan commented 5 months ago

Thank you for the feedback @pamelafox . We will investigate and get back to you asap.

cc @azureml-github