AML Environment creation is throwing an error in restclient without earlier errors during setup (using 'build' with dockerfile link)

[juanvergaramunoz]

• Package Name: azure.ai.ml
• Package Version: 1.4.0
• Operating System: windows
• Python Version: 3.7.16

Describe the bug Following the latest guidance to create an environment through MLClient* raises a ContextURI issue in a subpackage that I am unable to debug. Environment creation runs without any issues. It is the creation of a new environment using the ml_client.environments.create_update(my_env) that is raising the issue.

To Reproduce Steps to reproduce the behavior:

1. Set up your AML workspace.
2. Set up you conda or python environment.
3. Run the following Python commands:

import os
from azure.ai.ml.entities import Environment, BuildContext
from azure.ai.ml import MLClient
from azure.identity import InteractiveBrowserCredential

credential = InteractiveBrowserCredential()
ml_client = MLClient.from_config(credential=credential, path='my_aml_config_filepath')
environment_dockerfile_path = os.path.join(os.getcwd(), 'my_dockerfile_filepath')
gpu_env = Environment(
    name='my_environment_name',
    build=BuildContext(dockerfile_path=ray_environment_dockerfile_path)
)
ray_gpu_env = ml_client.environments.create_or_update(gpu_env)

Expected behavior An error should be thrown earlier if the submittal pipeline is wrong. Otherwise, the environment creation should succeed.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Full stack trace for an example:

Traceback (most recent call last):
  File "core_aml_pipeline_class.py", line 214, in <module>
    aml_pipeline = AML_Pipeline(**job_config)
  File "core_aml_pipeline_class.py", line 41, in __init__
    self._environment_setup()
  File "core_aml_pipeline_class.py", line 195, in _environment_setup
    ray_gpu_env = aml_environment_create(self.ml_client, self.ray_environment_name, dockerfile_path)
  File "C:\industrial-metaverse-private\project-nobo\examples\gymnasium_integration\aml_utils\aml_codeflow.py", line 22, in aml_environment_create
    ray_gpu_env = ml_client.environments.create_or_update(ray_gpu_env)
  File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\ai\ml\operations\_environment_operations.py", line 148, in create_or_update
    raise ex
  File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\ai\ml\operations\_environment_operations.py", line 138, in create_or_update
    **self._kwargs,
  File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\core\tracing\decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\ai\ml\_restclient\v2022_05_01\operations\_environment_versions_operations.py", line 520, in create_or_update
    raise HttpResponseError(response=response, model=error, error_format=ARMErrorFormat)
azure.core.exceptions.HttpResponseError: (UserError) Request is invalid and/or missing fields.
Code: UserError
Message: Request is invalid and/or missing fields.
Exception Details:      (RequestInvalid) The ContextUri field is required.
        Code: RequestInvalid
        Message: The ContextUri field is required.
Additional Information:Type: ComponentName
Info: {
    "value": "managementfrontend"
}Type: Correlation
Info: {
    "value": {
        "operation": "a94eca43244498401f51b918af4acfe2",
        "request": "b1b0cf12a95f5ea3"
    }
}Type: Environment
Info: {
    "value": "eastus2"
}Type: Location
Info: {
    "value": "eastus2"
}Type: Time
Info: {
    "value": "2023-03-10T15:52:37.5535658+00:00"
}Type: InnerError
Info: {
    "value": {
        "code": "RequestInvalid",
        "innerError": null
    }
}Type: MessageFormat
Info: {
    "value": "Request is invalid and/or missing fields."
}

[kashifkhan]

Thank you for the feedback @juanvergaramunoz . We will investigate and get back to you asap.

cc @azureml-github

[hswang-wang]

@juanvergaramunoz It seems like in your code, BuildContext is missing a path parameter. build=BuildContext(dockerfile_path=ray_environment_dockerfile_path) Could you add path parameter and try it again?

[juanvergaramunoz]

Thanks for your review @hswang-wang. Note, though as per documentation: BuildContext takes EITHER argument path or dockerfile_path:

• https://learn.microsoft.com/en-us/python/api/azure-ai-ml/azure.ai.ml.entities.buildcontext?view=azure-python

I am using dockerfile_path since I wanted to specify a pointer to the specific dockerfile name to be used for setting up the image. Otherwise, I need to rename the file to Dockerfile each time. Moreover, the command doesn't raise any issues when run.

[hswang-wang]

@juanvergaramunoz I was able to repro the issue. I'm reaching out to the service team to confirm is ContextUri required when creating an environment

[juanvergaramunoz]

Appreciate your prompt response. Thanks a lot @hswang-wang -- Feel free to ping me at any time if there are any tests that I can help you with on my side --

[hswang-wang]

@juanvergaramunoz Thanks for your patience. I confirmed that path parameter is required in BuildContext. path is mapping to ContextUri in BuildContext and ContextUri is required, so it seems if we're missing path, we will run into the error saying ContextUri is required. Apologize that there's gap in the documentation between what's required and what's not. Please try again with adding the path parameter and let me know if you still run into the issue. In the meantime, we will update the documentation.

[juanvergaramunoz]

Thanks @hswang-wang -- that resolved my issue. Shall I open a pull request on azure ai ml to trigger an error if no path attribute is parsed? Or is that requirement only a problem for this pipeline?

I am happy to help making it clearer for future coders that might encounter the same issue - please advise. Otherwise, we are good to close this. Thanks a lot for your prompt help, truly appreciate it!

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github, @Azure/azure-ml-sdk.

Issue Details

---

- **Package Name**: azure.ai.ml - **Package Version**: 1.4.0 - **Operating System**: windows - **Python Version**: 3.7.16 **Describe the bug** Following the latest guidance to create an environment through MLClient* raises a ContextURI issue in a subpackage that I am unable to debug. Environment creation runs without any issues. It is the creation of a new environment using the ml_client.environments.create_update(my_env) that is raising the issue. **To Reproduce** Steps to reproduce the behavior: 1. Set up your AML workspace. 2. Set up you conda or python environment. 3. Run the following Python commands: ``` import os from azure.ai.ml.entities import Environment, BuildContext from azure.ai.ml import MLClient from azure.identity import InteractiveBrowserCredential credential = InteractiveBrowserCredential() ml_client = MLClient.from_config(credential=credential, path='my_aml_config_filepath') environment_dockerfile_path = os.path.join(os.getcwd(), 'my_dockerfile_filepath') gpu_env = Environment( name='my_environment_name', build=BuildContext(dockerfile_path=ray_environment_dockerfile_path) ) ray_gpu_env = ml_client.environments.create_or_update(gpu_env) ``` **Expected behavior** An error should be thrown earlier if the submittal pipeline is wrong. Otherwise, the environment creation should succeed. **Screenshots** If applicable, add screenshots to help explain your problem. **Additional context** Full stack trace for an example: ``` Traceback (most recent call last): File "core_aml_pipeline_class.py", line 214, in aml_pipeline = AML_Pipeline(**job_config) File "core_aml_pipeline_class.py", line 41, in __init__ self._environment_setup() File "core_aml_pipeline_class.py", line 195, in _environment_setup ray_gpu_env = aml_environment_create(self.ml_client, self.ray_environment_name, dockerfile_path) File "C:\industrial-metaverse-private\project-nobo\examples\gymnasium_integration\aml_utils\aml_codeflow.py", line 22, in aml_environment_create ray_gpu_env = ml_client.environments.create_or_update(ray_gpu_env) File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\ai\ml\operations\_environment_operations.py", line 148, in create_or_update raise ex File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\ai\ml\operations\_environment_operations.py", line 138, in create_or_update **self._kwargs, File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\core\tracing\decorator.py", line 78, in wrapper_use_tracer return func(*args, **kwargs) File "C:\Users\juanvergara\Anaconda3\envs\aml_rllib_test3\lib\site-packages\azure\ai\ml\_restclient\v2022_05_01\operations\_environment_versions_operations.py", line 520, in create_or_update raise HttpResponseError(response=response, model=error, error_format=ARMErrorFormat) azure.core.exceptions.HttpResponseError: (UserError) Request is invalid and/or missing fields. Code: UserError Message: Request is invalid and/or missing fields. Exception Details: (RequestInvalid) The ContextUri field is required. Code: RequestInvalid Message: The ContextUri field is required. Additional Information:Type: ComponentName Info: { "value": "managementfrontend" }Type: Correlation Info: { "value": { "operation": "a94eca43244498401f51b918af4acfe2", "request": "b1b0cf12a95f5ea3" } }Type: Environment Info: { "value": "eastus2" }Type: Location Info: { "value": "eastus2" }Type: Time Info: { "value": "2023-03-10T15:52:37.5535658+00:00" }Type: InnerError Info: { "value": { "code": "RequestInvalid", "innerError": null } }Type: MessageFormat Info: { "value": "Request is invalid and/or missing fields." } ```

Author:	juanvergaramunoz
Assignees:	luigiw
Labels:	`question`, `Machine Learning`, `Service Attention`, `customer-reported`
Milestone:	-

[xiangyan99]

@hswang-wang any updates?

[clarissesimoes]

+1 on this. I'm seeing the same issue

[Uranium2]

Had the same issue. I had to add path. Please update the documentation that says that path will be uploaded to Azure with the dockerfile specified. And ALSO, if you have a BuildContext, you can not use conda_file argument in Environment and you must do it direclty inside the dockerfile.

That's a regression with V1

[achauhan-scc]

path is marked as required in documentation