Python 3.11 image

[gjurdzinski-deepsense]

Is there a plan to provide an image with python3.11?

[vizhur]

Base images are not python specific. Conda is preinstalled in every image that allows you to create an isolated environment of the interpreter version you need for the derived image. Base environment interpreter version is irrelevant and following Anaconda recommendations, we do not recommend to install packages or any other use of the base conda environment.

[chrissydonaldson]

@gjurdzinski-deepsense Take a look at curated environments within AzureML, for example mcr.microsoft.com/azureml/curated/minimal-py311-inference:6

[javigsg10]

I'm experiencing dependency issues when trying to create a conda environment for a model that requires Python 3.11 in Azure ML. Despite specifying Python 3.11 in my environment configuration, the dependency conflicts continue to block successful setup.

Environment Details: Azure ML Service: Azure Machine Learning Python Version Required: 3.11 or higher Images Tried:

• mcr.microsoft.com/azureml/curated/minimal-py311-inference:6
• mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest

When using the images listed above, I encounter several dependency errors during the environment setup, especially with packages like ruamel.yaml which fail due to header mismatches and compatibility issues in the build process.

Is there a prebuilt Azure ML image with Python 3.11 already configured that can bypass these dependency conflicts?

[vizhur]

Your dependencies and build log would help to understand your issue. Though it is unlikely it is related to the base images, I would suggest to post that question on stackoverflow

[javigsg10]

conda.yaml

channels:
- conda-forge
dependencies:
- python=3.11.9
- ruamel.yaml==0.17.21
- pip=24.3.1
- numpy=1.26.*
- scipy=1.11.4
- cython=3.0.10
- pip:
  - azureml-defaults
  - azureml-inference-server-http
  - joblib>=1.2,<1.4
  - numpy==1.26.*
  - pandas==2.1.*
  - scikit-learn==1.3.*
  - scipy>=1.11.4
  - werkzeug>=3.0.3
  - lale[fairness]>=0.8,<0.9
  - autoai-libs==2.0.*
  - azureml-dataprep==5.3.0
  - meson-python>=0.16.0
  - cython>=3.0.10
  - lightgbm==4.2.*
  - snapml>=1.14.0,<1.15.0
  - xgboost==2.0.*

env = Environment.from_conda_specification(name='autoai-env', file_path='./conda.yaml')
env.docker.enabled = True
env.docker.base_image = 'mcr.microsoft.com/azureml/curated/minimal-py311-inference:6' #mcr.microsoft.com/azureml/curated/minimal-py311-inference:6

])

@vizhur could you please let me know if this approach is correct for installing libraries that strictly need python3.11 or above, or do you see anything strange?

thanks

[vizhur]

base image interpreter version doesn't matter here. Isolated environment will be created from your spec with interpreter version you specified. If you really need ruamel.yaml instelled, I would try to have it in pip dependencies, though I think it might already be a nested dependency under azureml-defaults/azureml-core