Binder environment solve failing

[kafitzgerald]

First, thanks so much for all of the great resources you've put together!

In trying to launch the tutorial on the cloud (using the launch binder link), I noticed the environment creation on Binder seems to be failing (see details below). It looks like the [file for the specific version of the] package of interest is marked broken on conda-forge.

``` Waiting for build to start... Picked Git content provider. Cloning into '/tmp/repo2dockerw2qrr8o4'... HEAD is now at 4e29164 [pre-commit.ci] pre-commit autoupdate (#253) Building conda environment for python=3.11 Using CondaBuildPack builder Step 1/50 : FROM docker.io/library/buildpack-deps:jammy ---> b6235acc32ce Step 2/50 : ENV DEBIAN_FRONTEND=noninteractive ---> Using cache ---> 755cd1b1e980 Step 3/50 : RUN apt-get -qq update && apt-get -qq install --yes --no-install-recommends locales > /dev/null && apt-get -qq purge && apt-get -qq clean && rm -rf /var/lib/apt/lists/* ---> Using cache ---> de9f0e4a05cd Step 4/50 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen ---> Using cache ---> 92f56f191f0c Step 5/50 : ENV LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 LANGUAGE=en_US.UTF-8 ---> Using cache ---> 183e604467e3 Step 6/50 : ENV SHELL=/bin/bash ---> Using cache ---> 29d870378c0e Step 7/50 : ARG NB_USER ---> Using cache ---> 3294b005ebab Step 8/50 : ARG NB_UID ---> Using cache ---> fb48a5f7c4f6 Step 9/50 : ENV USER=${NB_USER} HOME=/home/${NB_USER} ---> Using cache ---> 6f69538c7a7f Step 10/50 : RUN groupadd --gid ${NB_UID} ${NB_USER} && useradd --comment "Default user" --create-home --gid ${NB_UID} --no-log-init --shell /bin/bash --uid ${NB_UID} ${NB_USER} ---> Using cache ---> eda0cac86361 Step 11/50 : RUN apt-get -qq update && apt-get -qq install --yes --no-install-recommends gettext-base less unzip > /dev/null && apt-get -qq purge && apt-get -qq clean && rm -rf /var/lib/apt/lists/* ---> Using cache ---> 1ff96132f912 Step 12/50 : EXPOSE 8888 ---> Using cache ---> 5a4faedfdb99 Step 13/50 : ENV APP_BASE=/srv ---> Using cache ---> 99cded126c26 Step 14/50 : ENV CONDA_DIR=${APP_BASE}/conda ---> Using cache ---> 2cdab4c7820c Step 15/50 : ENV NB_PYTHON_PREFIX=${CONDA_DIR}/envs/notebook ---> Using cache ---> 1fe020df2668 Step 16/50 : ENV NPM_DIR=${APP_BASE}/npm ---> Using cache ---> 860da8dad357 Step 17/50 : ENV NPM_CONFIG_GLOBALCONFIG=${NPM_DIR}/npmrc ---> Using cache ---> 98eea01df937 Step 18/50 : ENV NB_ENVIRONMENT_FILE=/tmp/env/environment.lock ---> Using cache ---> 8cbdc0f7937a Step 19/50 : ENV MAMBA_ROOT_PREFIX=${CONDA_DIR} ---> Using cache ---> 73d9bc52ee1b Step 20/50 : ENV MAMBA_EXE=${CONDA_DIR}/bin/mamba ---> Using cache ---> 02bf661ccfc9 Step 21/50 : ENV CONDA_PLATFORM=linux-64 ---> Using cache ---> e84d65f49470 Step 22/50 : ENV KERNEL_PYTHON_PREFIX=${NB_PYTHON_PREFIX} ---> Using cache ---> f3920c4a7f2a Step 23/50 : ENV PATH=${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH} ---> Using cache ---> b4e06e806525 Step 24/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-e70a7b /etc/profile.d/activate-conda.sh ---> Using cache ---> 7de2973d9eb0 Step 25/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e11-2dlinux-2d64-2elock-4045c1 /tmp/env/environment.lock ---> Using cache ---> d5e0338e1261 Step 26/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dbase-2denv-2ebash-6a6072 /tmp/install-base-env.bash ---> Using cache ---> 8b966e71692c Step 27/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-base-env.bash' && rm -rf /tmp/install-base-env.bash /tmp/env ---> Using cache ---> c02f05a98b6e Step 28/50 : RUN mkdir -p ${NPM_DIR} && chown -R ${NB_USER}:${NB_USER} ${NPM_DIR} ---> Using cache ---> c4693ba673c1 Step 29/50 : USER root ---> Using cache ---> 9ef8ec836ad9 Step 30/50 : ARG REPO_DIR=${HOME} ---> Using cache ---> 9bdf2484928b Step 31/50 : ENV REPO_DIR=${REPO_DIR} ---> Using cache ---> d27859b6f602 Step 32/50 : RUN if [ ! -d "${REPO_DIR}" ]; then /usr/bin/install -o ${NB_USER} -g ${NB_USER} -d "${REPO_DIR}"; fi ---> Using cache ---> 1046374f9614 Step 33/50 : WORKDIR ${REPO_DIR} ---> Using cache ---> b45884b871ce Step 34/50 : RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR} ---> Using cache ---> b830b434c664 Step 35/50 : ENV PATH=${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH} ---> Using cache ---> 74f4cdfb8892 Step 36/50 : ENV CONDA_DEFAULT_ENV=${KERNEL_PYTHON_PREFIX} ---> Using cache ---> 12e1a5c324a3 Step 37/50 : COPY --chown=1000:1000 src/.binder/environment.yml ${REPO_DIR}/.binder/environment.yml ---> Using cache ---> ef3ef735f463 Step 38/50 : USER ${NB_USER} ---> Using cache ---> 61bad9173e36 Step 39/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file ".binder/environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} ' ---> Running in 292066c511c8 Channels: - conda-forge - defaults Platform: linux-64 Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... failed PackagesNotFoundError: The following packages are not available from current channels: - rdma-core==50.0=hd3aeb46_0 Current channels: - https://conda.anaconda.org/conda-forge/linux-64 - https://repo.anaconda.com/pkgs/main/linux-64 - https://repo.anaconda.com/pkgs/r/linux-64 To search for alternate channels that may provide the conda package you're looking for, navigate to https://anaconda.org and use the search bar at the top of the page. time: 60.367 ---> Removed intermediate container 9d18d00af3eb The command '/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_Pment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '' retur ```

[scottyhq]

PackagesNotFoundError: The following packages are not available from current channels:

  - rdma-core==50.0=hd3aeb46_0

Thanks for pointing this out @kafitzgerald. Looks like this rdma package is a transitive dependency of libarrow (from mamba repoquery whoneeds rdma-core)

Hopefully just bumping to more recent versions of pyarrow fixes this, until then it seems running via codespaces still works b/c we use a lockfile for that and so it isn't aware of packages being later labeled as "broken" https://github.com/xarray-contrib/xarray-tutorial?tab=readme-ov-file#github-codespaces

[scottyhq]

huh... turns out repo2docker (& therefore mybinder.org doesn't support python 3.12 yet) https://github.com/jupyterhub/repo2docker/issues/1225