Cannot build cell in python flavor that needs rasterio

[skoulouzis]

Using this Dockerfile (tryed all versions of naavre-cell-build-python: v0.17, v0.11,v0.5,v0.2 ):

FROM ghcr.io/qcdis/naavre/naavre-cell-build-python:v0.17 AS build
COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yaml .
RUN micromamba install -y -n venv -f environment.yaml
ARG MAMBA_DOCKERFILE_ACTIVATE=1
USER root
RUN conda-pack -p /opt/conda/envs/venv -o /tmp/env.tar && \
    mkdir /venv && cd /venv && tar xf /tmp/env.tar && \
    rm /tmp/env.tar
RUN /venv/bin/conda-unpack

FROM jupyter/base-notebook AS runtime
COPY --from=build /venv /venv
WORKDIR /app
COPY . .

The environment.yaml:

name: venv
channels:
  - conda-forge
dependencies:
  - pip
  - python>=3.8
  - nbconvert
  - requests
  - matplotlib
  - rasterio
  - papermill
  - ipykernel

We get an error:

 => CACHED [build 3/5] RUN micromamba install -y -n venv -f environment.yaml                                                                                             0.0s
 => ERROR [build 4/5] RUN conda-pack -p /opt/conda/envs/venv -o /tmp/env.tar &&     mkdir /venv && cd /venv && tar xf /tmp/env.tar &&     rm /tmp/env.tar                0.5s
------                                                                                                                                                                        
 > [build 4/5] RUN conda-pack -p /opt/conda/envs/venv -o /tmp/env.tar &&     mkdir /venv && cd /venv && tar xf /tmp/env.tar &&     rm /tmp/env.tar:                           
0.409 CondaPackError:                                                                                                                                                         
0.409 Files managed by conda were found to have been deleted/overwritten in the                                                                                               
0.409 following packages:                                                                                                                                                     
0.409                                                                                                                                                                         
0.409 - libstdcxx 14.1.0:
0.409     lib/libstdc++.so
0.409     lib/libstdc++.so.6
0.409     lib/libstdc++.so.6.0.33
0.409     share/licenses/libstdc++/RUNTIME.LIBRARY.EXCEPTION
0.409 - libgcc 14.1.0:
0.409     lib/libatomic.so
0.409     lib/libatomic.so.1
0.409     lib/libatomic.so.1.2.0
0.409     + 11 others
0.409 
0.409 This is usually due to `pip` uninstalling or clobbering conda managed files,
0.409 resulting in an inconsistent environment. Please check your environment for
0.409 conda/pip conflicts using `conda list`, and fix the environment by ensuring
0.409 only one version of each package is installed (conda preferred).
0.409 Collecting packages...
------
Dockerfile:16
--------------------
  15 |     USER root
  16 | >>> RUN conda-pack -p /opt/conda/envs/venv -o /tmp/env.tar && \
  17 | >>>     mkdir /venv && cd /venv && tar xf /tmp/env.tar && \
  18 | >>>     rm /tmp/env.tar

If we remove rasterio from the environment.yaml the build is successful

However, if we use the ghcr.io/qcdis/naavre/naavre-cell-build-python base docker together:

FROM mambaorg/micromamba:1.5.6 AS build
RUN micromamba install -y -n base -c conda-forge conda-pack
ARG CONDA_ENV_FILE
COPY p-environment.yaml p-environment.yaml
RUN micromamba create -y -n venv -f p-environment.yaml && \
    micromamba clean --all --yes

COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yaml .
RUN micromamba install -y -n venv -f environment.yaml
ARG MAMBA_DOCKERFILE_ACTIVATE=1
USER root
RUN conda-pack -p /opt/conda/envs/venv -o /tmp/env.tar && \
    mkdir /venv && cd /venv && tar xf /tmp/env.tar && \
    rm /tmp/env.tar
RUN /venv/bin/conda-unpack

FROM jupyter/base-notebook AS runtime
COPY --from=build /venv /venv
WORKDIR /app
COPY . .

with the p-environment.yaml:

name: venv
channels:
  - conda-forge
dependencies:
  - python>=3.9
  - pip
  - git

the build is successful

[skoulouzis]

If we remove libgcc build is successful:

FROM ghcr.io/qcdis/naavre/naavre-cell-build-python:v0.17 AS build

RUN micromamba list -n base | grep libgcc
RUN micromamba list -n venv | grep libgcc

COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yaml .
RUN micromamba install -y -n venv -f environment.yaml

RUN micromamba remove -y -n venv libgcc

ARG MAMBA_DOCKERFILE_ACTIVATE=1
USER root
RUN conda-pack -p /opt/conda/envs/venv -o /tmp/env.tar && \
    mkdir /venv && cd /venv && tar xf /tmp/env.tar && \
    rm /tmp/env.tar
RUN /venv/bin/conda-unpack

FROM jupyter/base-notebook AS runtime
COPY --from=build /venv /venv
WORKDIR /app
COPY . .

[skoulouzis]

To get the libgcc version installed on the ghcr.io/qcdis/naavre/naavre-cell-build-python:v0.17 we run:

docker run -it ghcr.io/qcdis/naavre/naavre-cell-build-python:v0.17 /bin/bash

In the ghcr.io/qcdis/naavre/naavre-cell-build-python:v0.17:

micromamba activate venv
micromamba list | grep libgcc
  _libgcc_mutex     0.1           conda_forge         conda-forge
  libgcc-ng         14.1.0        h77fa898_0          conda-forge

After installing rasterio:

micromamba list | grep libgcc
  _libgcc_mutex              0.1           conda_forge          conda-forge
  libgcc                     14.1.0        h77fa898_1           conda-forge
  libgcc-ng                  14.1.0        h69a702a_1           conda-forge

[skoulouzis]

Running :

docker run -it mambaorg/micromamba:1.5.6 /bin/bash

In mambaorg/micromamba:1.5.6 Install git:

micromamba create -n venv
micromamba activate venv
micromamba install -c conda-forge git

Get libgcc

micromamba list | grep libgcc
  _libgcc_mutex    0.1           conda_forge       conda-forge
  libgcc           14.1.0        h77fa898_1        conda-forge
  libgcc-ng        14.1.0        h69a702a_1        conda-forge

Install rasterio:

micromamba install -c conda-forge rasterio
micromamba list | grep libgcc
  _libgcc_mutex              0.1           conda_forge          conda-forge
  libgcc                     14.1.0        h77fa898_1           conda-forge
  libgcc-ng                  14.1.0        h69a702a_1           conda-forge

[gpelouze]

Fixed by re-building the base images