new clone today doesn't run generate - AttributeError: module 'azureml.metrics.constants' has no attribute 'QUESTION_ANSWERING'

[diberry]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. open dev container
2. update .env with values
3. update example_config with values
4. run python3 -m scripts generate --output=example_input/qa.jsonl --numquestions=14 --persource=2

vscode ➜ /workspaces/ai-rag-chat-evaluator $ python3 -m scripts generate --output=example_input/qa.jsonl --numquestions=14 --
persource=2
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/workspaces/ai-rag-chat-evaluator/scripts/__main__.py", line 3, in <module>
    from .cli import app
  File "/workspaces/ai-rag-chat-evaluator/scripts/cli.py", line 8, in <module>
    from .evaluate import run_evaluate_from_config
  File "/workspaces/ai-rag-chat-evaluator/scripts/evaluate.py", line 8, in <module>
    from azure.ai.generative.evaluate import evaluate
  File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/__init__.py", line 8, in <module>
    from ._evaluate import evaluate
  File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 22, in <module>
    from azure.ai.generative.evaluate._metric_handler import MetricHandler
  File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/_metric_handler.py", line 6, in <module>
    from azure.ai.generative.evaluate._constants import TYPE_TO_KWARGS_MAPPING
  File "/home/vscode/.local/lib/python3.11/site-packages/azure/ai/generative/evaluate/_constants.py", line 13, in <module>
    QA: constants.QUESTION_ANSWERING,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'azureml.metrics.constants' has no attribute 'QUESTION_ANSWERING'

Any log messages given by the failure

Expected/desired behavior

no errors

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

tried both win an mac running container

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[pamelafox]

Looks like an azureml-metrics release today broke the evaluate SDK. Filed here: https://github.com/Azure/azure-sdk-for-python/issues/33890

For now, I'll pin it:

azureml-metrics[generative-ai]==0.0.42