I am trying to construct and run pipelines from AzureML using the Python SDK v1 from a Python script but run into the following error:
Traceback (most recent call last): File "pipeline.py", line 200, in <module> pipeline_run.wait_for_completion() File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/pipeline/core/run.py", line 289, in wait_for_completion for step_run in children: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/core/run.py", line 467, in _rehydrate_runs for run_dto in run_dtos: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py", line 425, in get_descendants for child in children: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/experiment_client.py", line 342, in _client_filter for run_dto in run_dtos: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 264, in _call_paginated_api paginated_dto = next_page.wait(awaiter_name="ApiPagination") File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_common/async_utils/async_task.py", line 58, in wait res = self._handler(self._future, self._logger) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_common/async_utils/async_task.py", line 16, in basic_handler return future.result() File "/anaconda/envs/azureml_py38/lib/python3.8/concurrent/futures/_base.py", line 439, in result return self.__get_result() File "/anaconda/envs/azureml_py38/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result raise self._exception File "/anaconda/envs/azureml_py38/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 333, in _execute_with_base_arguments return ClientBase._execute_func_internal( File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 367, in _execute_func_internal left_retry = cls._handle_retry(back_off, left_retry, total_retry, error, logger, func) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 399, in _handle_retry raise error File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 358, in _execute_func_internal response = func(*args, **kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/operations/run_operations.py", line 107, in get_child raise models.ErrorResponseException(self._deserialize, response) azureml._restclient.models.error_response.ErrorResponseException: (ServiceError) Received 403 from a service request
The pipeline starts running just fine in the background but the terminal does not generate any logs following the error.
In addition to that, the pipeline fails at the AutoMLStep for the following reason:
Received 403 from a service request-{
"error": {
"code": "UserError",
"severity": null,
"message": "User is not authorized to access provided resources due to attempted access of the workspace data plane operations outside of the designated private link..",
These pipelines used to run fine until yesterday. Is there any change in behavior expected in the AzureML SDK v1?
Thanks.
I am trying to construct and run pipelines from AzureML using the Python SDK v1 from a Python script but run into the following error:
Traceback (most recent call last): File "pipeline.py", line 200, in <module> pipeline_run.wait_for_completion() File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/pipeline/core/run.py", line 289, in wait_for_completion for step_run in children: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/core/run.py", line 467, in _rehydrate_runs for run_dto in run_dtos: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py", line 425, in get_descendants for child in children: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/experiment_client.py", line 342, in _client_filter for run_dto in run_dtos: File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 264, in _call_paginated_api paginated_dto = next_page.wait(awaiter_name="ApiPagination") File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_common/async_utils/async_task.py", line 58, in wait res = self._handler(self._future, self._logger) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_common/async_utils/async_task.py", line 16, in basic_handler return future.result() File "/anaconda/envs/azureml_py38/lib/python3.8/concurrent/futures/_base.py", line 439, in result return self.__get_result() File "/anaconda/envs/azureml_py38/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result raise self._exception File "/anaconda/envs/azureml_py38/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 333, in _execute_with_base_arguments return ClientBase._execute_func_internal( File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 367, in _execute_func_internal left_retry = cls._handle_retry(back_off, left_retry, total_retry, error, logger, func) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 399, in _handle_retry raise error File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/clientbase.py", line 358, in _execute_func_internal response = func(*args, **kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/_restclient/operations/run_operations.py", line 107, in get_child raise models.ErrorResponseException(self._deserialize, response) azureml._restclient.models.error_response.ErrorResponseException: (ServiceError) Received 403 from a service request
The pipeline starts running just fine in the background but the terminal does not generate any logs following the error. In addition to that, the pipeline fails at the AutoMLStep for the following reason:
These pipelines used to run fine until yesterday. Is there any change in behavior expected in the AzureML SDK v1? Thanks.