add_pip_package does not update pip requirements of environments

I am using the following code to create a custom environment by cloning and modifying a curated environment.


pip_libs = ["tensorboard", "torchvision", "tqdm", "terminaltables","easydict", "SimpleITK", "pydicom", "python-snappy", "scikit-image", "scikit-learn", "h5py", "pandas", "pillow", "opencv-python", "timm", "torchinfo"]

myenv = Environment.get(ws, "AzureML-pytorch-1.10-ubuntu18.04-py38-cuda11-gpu").clone("new_env")
conda_dep = myenv.python.conda_dependencies

for pl in pip_libs:
     conda_dep.add_pip_package(pl)

conda_dep.add_conda_package("cmake")
myenv.python.conda_dependencies=conda_dep

myenv.register(ws)

None of these new pip packages get added to the new environment. Inspecting the dockerfile associated with this newly created environment confirms this:

FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.1-cudnn8-ubuntu18.04:20230103.v1

ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/pytorch-1.10
# Create conda environment
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
    python=3.8 \
    pip=22.1.2 \
    pytorch=1.10.0 \
    torchvision=0.11.1 \
    torchaudio=0.10.0 \
    cudatoolkit=11.1.1 \
    nvidia-apex=0.1.0 \
    gxx_linux-64 \
    -c anaconda -c pytorch -c conda-forge

# Prepend path to AzureML conda environment
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH

# Install pip dependencies
RUN pip install 'matplotlib>=3.3,<3.4' \
                'psutil>=5.8,<5.9' \
                'tqdm>=4.59,<4.63' \
                'pandas>=1.3,<1.4' \
                'scipy>=1.5,<1.8' \
                'numpy>=1.10,<1.22' \
                'ipykernel~=6.0' \
                # upper bound azure-core to address typing-extensions conflict
                'azure-core<1.23.0' \
                'azureml-core==1.48.0' \
                'azureml-defaults==1.48.0' \
                'azureml-mlflow==1.48.0' \
                'azureml-telemetry==1.48.0' \
                'tensorboard==2.6.0' \
                'tensorflow-gpu==2.6.0' \
                'onnxruntime-gpu>=1.7,<1.10' \
                'horovod==0.23' \
                'future==0.18.2' \
                'torch-tb-profiler==0.3.1' \
                'debugpy~=1.6.3'

# This is needed for mpi to locate libpython
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH

It seems like nothing was added to the base environment that is cloned and modified.

print("SDK version:", azureml.core.VERSION)
'1.48.0'

Azure / MachineLearningNotebooks

add_pip_package does not update pip requirements of environments #1888