search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.28k stars 21.45k forks source link

Cannot run example in Azure AI Python SDK #120397

Closed mattans closed 3 months ago

mattans commented 8 months ago

I'm trying to make an AIClient object like specified in the page, but I'm getting this error:

Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from azure.identity import DefaultAzureCredential
>>> from azure.ai.resources.client import AIClient
>>> credential = DefaultAzureCredential()
>>> client = AIClient.from_config(credential=credential)
Class AIClient: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/resources/client/_ai_client.py", line 140, in from_config
    return AIClient(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/ml/_utils/_experimental.py", line 67, in wrapped
    return func(*args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/resources/client/_ai_client.py", line 92, in __init__
    ai_resource_name = ai_resource_name or self._ml_client.workspaces.get(project_name).workspace_hub.split("/")[-1]
AttributeError: 'NoneType' object has no attribute 'split'

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

RamanathanChinnappan-MSFT commented 8 months ago

@mattans Thanks for your feedback! We will investigate and update as appropriate.

mattans commented 8 months ago

I was able to get through this, with replacing "AIClient.from_config" with "AIClient", and adding "ai_resource_name":

client = AIClient(
    credential=credential,
    subscription_id=*****
    resource_group_name=*****
    project_name=*****

    # code fails if not passing this!
    # ai_resource_name = ai_resource_name or self._ml_client.workspaces.get(project_name).workspace_hub.split("/")[-1]
    # AttributeError: 'NoneType' object has no attribute 'split'
    ai_resource_name="BLA_BLA",  
)

Will try to run the rest of the code now.

mattans commented 8 months ago

I'm now getting a 340-lines stacktrace:

(azureml_py38) azureuser@maserry-cpu:~/cloudfiles/code/Users/maserry/playground$ python evaluate_with_azure_ai_python_sdk.py 
Keyring method: keyrings.alt.file.PlaintextKeyring (priority: 0.5)
Class AIClient: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py:202: UserWarning: y_pred is deprecated, please use "answer" instead
  warnings.warn("y_pred is deprecated, please use \"answer\" instead")
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py:206: UserWarning: y_test is deprecated, please use "ground_truth" instead
  warnings.warn("y_test is deprecated, please use \"ground_truth\" instead")
[2024-03-04 19:12:58,704][promptflow._sdk._submitter.run_submitter][INFO] - Submitting run qa_variant_0_20240304_191258_405893, log path: /home/azureuser/.promptflow/.runs/qa_variant_0_20240304_191258_405893/logs.txt
Keyring method: keyrings.alt.file.PlaintextKeyring (priority: 0.5)
Class AIClient: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py:202: UserWarning: y_pred is deprecated, please use "answer" instead
  warnings.warn("y_pred is deprecated, please use \"answer\" instead")
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py:206: UserWarning: y_test is deprecated, please use "ground_truth" instead
  warnings.warn("y_test is deprecated, please use \"ground_truth\" instead")
[2024-03-04 19:13:05,578][promptflow._sdk._submitter.run_submitter][INFO] - Submitting run qa_variant_0_20240304_191305_294313, log path: /home/azureuser/.promptflow/.runs/qa_variant_0_20240304_191305_294313/logs.txt
[2024-03-04 19:13:07,397][promptflow._sdk._submitter.run_submitter][WARNING] - Run qa_variant_0_20240304_191305_294313 failed when executing in executor with exception Unexpected error occurred while executing the batch run. Error: (RuntimeError) 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable...
Traceback (most recent call last):
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 205, in run
    return async_run_allowing_running_loop(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop
    return asyncio.run(async_func(*args, **kwargs))
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 320, in _exec_in_task
    return task.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 349, in _exec
    results, is_timeout = self._executor_proxy._exec_batch(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_python_executor_proxy.py", line 72, in _exec_batch
    with LineExecutionProcessPool(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 185, in __enter__
    self._processes_manager.start_processes()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_process_manager.py", line 257, in start_processes
    process.start()
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 359, in _evaluate
    inbuilt_metrics_results = inbuilt_metrics_handler.calculate_metrics()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_metric_handler.py", line 108, in calculate_metrics
    pf_run = run_pf_flow_with_dict_list(flow_path, dict_list, flow_params={"connections": {node: connection_override for node in nodes_list}})
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_utils.py", line 65, in run_pf_flow_with_dict_list
    return pf_client.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_pf_client.py", line 144, in run
    return self.runs.create_or_update(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_telemetry/activity.py", line 245, in wrapper
    return f(self, *args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/operations/_run_operations.py", line 103, in create_or_update
    created_run = RunSubmitter(client=self._client).submit(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 39, in submit
    self._run_bulk(run=run, stream=stream, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 79, in _run_bulk
    self._submit_bulk_run(flow=flow, run=run, local_storage=local_storage)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 158, in _submit_bulk_run
    raise e
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 126, in _submit_bulk_run
    batch_result = batch_engine.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 228, in run
    raise unexpected_error from e
promptflow._core._errors.UnexpectedError: Unexpected error occurred while executing the batch run. Error: (RuntimeError) 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable..

Traceback (most recent call last):
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 205, in run
    return async_run_allowing_running_loop(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop
    return asyncio.run(async_func(*args, **kwargs))
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 320, in _exec_in_task
    return task.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 349, in _exec
    results, is_timeout = self._executor_proxy._exec_batch(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_python_executor_proxy.py", line 72, in _exec_batch
    with LineExecutionProcessPool(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 185, in __enter__
    self._processes_manager.start_processes()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_process_manager.py", line 257, in start_processes
    process.start()
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/maserry-cpu/code/Users/maserry/playground/evaluate_with_azure_ai_python_sdk.py", line 28, in <module>
    result = evaluate(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/core/tracing/decorator.py", line 76, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper
    return f(*args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 237, in evaluate
    evaluation_result = _evaluate(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 359, in _evaluate
    inbuilt_metrics_results = inbuilt_metrics_handler.calculate_metrics()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_metric_handler.py", line 108, in calculate_metrics
    pf_run = run_pf_flow_with_dict_list(flow_path, dict_list, flow_params={"connections": {node: connection_override for node in nodes_list}})
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_utils.py", line 65, in run_pf_flow_with_dict_list
    return pf_client.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_pf_client.py", line 144, in run
    return self.runs.create_or_update(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_telemetry/activity.py", line 245, in wrapper
    return f(self, *args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/operations/_run_operations.py", line 103, in create_or_update
    created_run = RunSubmitter(client=self._client).submit(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 39, in submit
    self._run_bulk(run=run, stream=stream, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 79, in _run_bulk
    self._submit_bulk_run(flow=flow, run=run, local_storage=local_storage)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 158, in _submit_bulk_run
    raise e
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 126, in _submit_bulk_run
    batch_result = batch_engine.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 228, in run
    raise unexpected_error from e
promptflow._core._errors.UnexpectedError: Unexpected error occurred while executing the batch run. Error: (RuntimeError) 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable..
[2024-03-04 19:13:09,657][promptflow._sdk._submitter.run_submitter][WARNING] - Run qa_variant_0_20240304_191258_405893 failed when executing in executor with exception Failed to start spawned fork process manager.
Traceback (most recent call last):
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 359, in _evaluate
    inbuilt_metrics_results = inbuilt_metrics_handler.calculate_metrics()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_metric_handler.py", line 108, in calculate_metrics
    pf_run = run_pf_flow_with_dict_list(flow_path, dict_list, flow_params={"connections": {node: connection_override for node in nodes_list}})
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_utils.py", line 65, in run_pf_flow_with_dict_list
    return pf_client.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_pf_client.py", line 144, in run
    return self.runs.create_or_update(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_telemetry/activity.py", line 245, in wrapper
    return f(self, *args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/operations/_run_operations.py", line 103, in create_or_update
    created_run = RunSubmitter(client=self._client).submit(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 39, in submit
    self._run_bulk(run=run, stream=stream, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 79, in _run_bulk
    self._submit_bulk_run(flow=flow, run=run, local_storage=local_storage)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 158, in _submit_bulk_run
    raise e
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 126, in _submit_bulk_run
    batch_result = batch_engine.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 218, in run
    raise e
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 205, in run
    return async_run_allowing_running_loop(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop
    return asyncio.run(async_func(*args, **kwargs))
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 320, in _exec_in_task
    return task.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 349, in _exec
    results, is_timeout = self._executor_proxy._exec_batch(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_python_executor_proxy.py", line 82, in _exec_batch
    line_results = pool.run(zip(line_number, batch_inputs))
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 546, in run
    async_result.get()
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 257, in _monitor_workers_and_process_tasks_in_thread
    index, process_id, process_name = self._get_process_info(index)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 205, in _get_process_info
    self._processes_manager.ensure_healthy()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_process_manager.py", line 295, in ensure_healthy
    raise ex
promptflow.executor._errors.SpawnedForkProcessManagerStartFailure: Failed to start spawned fork process manager

Traceback (most recent call last):
  File "evaluate_with_azure_ai_python_sdk.py", line 28, in <module>
    result = evaluate(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/core/tracing/decorator.py", line 76, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper
    return f(*args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 237, in evaluate
    evaluation_result = _evaluate(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_evaluate.py", line 359, in _evaluate
    inbuilt_metrics_results = inbuilt_metrics_handler.calculate_metrics()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_metric_handler.py", line 108, in calculate_metrics
    pf_run = run_pf_flow_with_dict_list(flow_path, dict_list, flow_params={"connections": {node: connection_override for node in nodes_list}})
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/generative/evaluate/_utils.py", line 65, in run_pf_flow_with_dict_list
    return pf_client.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_pf_client.py", line 144, in run
    return self.runs.create_or_update(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_telemetry/activity.py", line 245, in wrapper
    return f(self, *args, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/operations/_run_operations.py", line 103, in create_or_update
    created_run = RunSubmitter(client=self._client).submit(run=run, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 39, in submit
    self._run_bulk(run=run, stream=stream, **kwargs)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 79, in _run_bulk
    self._submit_bulk_run(flow=flow, run=run, local_storage=local_storage)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 158, in _submit_bulk_run
    raise e
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_sdk/_submitter/run_submitter.py", line 126, in _submit_bulk_run
    batch_result = batch_engine.run(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 218, in run
    raise e
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 205, in run
    return async_run_allowing_running_loop(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop
    return asyncio.run(async_func(*args, **kwargs))
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/anaconda/envs/azureml_py38/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 320, in _exec_in_task
    return task.result()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_batch_engine.py", line 349, in _exec
    results, is_timeout = self._executor_proxy._exec_batch(
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/batch/_python_executor_proxy.py", line 82, in _exec_batch
    line_results = pool.run(zip(line_number, batch_inputs))
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 546, in run
    async_result.get()
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/anaconda/envs/azureml_py38/lib/python3.8/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 257, in _monitor_workers_and_process_tasks_in_thread
    index, process_id, process_name = self._get_process_info(index)
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_line_execution_process_pool.py", line 205, in _get_process_info
    self._processes_manager.ensure_healthy()
  File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/promptflow/executor/_process_manager.py", line 295, in ensure_healthy
    raise ex
promptflow.executor._errors.SpawnedForkProcessManagerStartFailure: Failed to start spawned fork process manager
RamanathanChinnappan-MSFT commented 8 months ago

@eric-urban could you please review this and update as appropriate.

mattans commented 8 months ago

I was able to resolve the multiprocessing error. Originally I placed the code you provided in a single script and just ran it, but that causes multiprocessing error, the code must start with freeze_support(), or better, if __name__ == "__main__"

Please consider adding to the documentation that the provided code has multiprocessing and must be invoked properly (when running in script mode, should add if __name__ == "__main__" at start, and when running in import mode, I don't know) Thanks!

RamanathanChinnappan-MSFT commented 8 months ago

@mattans I've delegated this to @eric-urban, a content author, to review and share their valuable insights.

eric-urban commented 3 months ago

Since the corresponding SDK was retired, I'll have to close this issue.

please-close