search

Azure / MachineLearningNotebooks

Python notebooks with ML and deep learning examples with Azure Machine Learning Python SDK | Microsoft
https://docs.microsoft.com/azure/machine-learning/service/
MIT License
4k stars 2.49k forks source link

Installing conda.yml on top of curated environment (GPU+PyTorch)? #1923

Closed movingabout closed 1 year ago

movingabout commented 1 year ago

I'd like to use a curated environment with GPUs and PyTorch (e.g. AzureML-pytorch-1.9-ubuntu18.04-py37-cuda11-gpu) and install python libraries based on a conda.yml on top.

How do I set this up correctly?

Ideally, I'd use the curated environment as a base image for a custom environment. But I'm having trouble setting the image parameter correctly. Is that even possible? 

environment = Environment(
    image= # ---> how do I address the curated environment?
    name=ENVIRONMENT_NAME,
    conda_file=CONDA_YML_PATH,
)
ml_client.environments.create_or_update(environment)

Thanks!

vizhur commented 1 year ago

image + conda spec will create an isolated environment on top of the base image with dependencies from your yml file. That defeats the purpose of using curated environment image as a base, as none of the python dependencies will be available. You can use docker context to install into active environment. Please note, pytorch 1.9 environment is deprecated and that partial environment resolution can result dependencies conflict.

Dockerfile:

` FROM mcr.microsoft.com/azureml/curated/acpt-pytorch-1.12-py38-cuda11.6-gpu:11

RUN pip install mycoolpackage `