[BUG] - Environments Accumulate Non-Reproducible States

[viniciusdc]

Context

The current Ansible task for managing conda environments gradually accumulates old packages, leading to non-reproducible states. This issue is particularly noticeable with deployments migrated from pre-2024.3.1 Nebari Slurm versions, where outdated packages such as Gator remain in the JupyterLab environments.

https://github.com/nebari-dev/nebari-slurm/blob/4ff70836391720b0a48d7e7d3a6ab954c576d541/roles/conda_environment/tasks/main.yaml#L2-L21

The mamba env update command in the task above only updates the existing environment without removing outdated packages that are no longer needed. Over time, this results in an environment that includes outdated dependencies, making it non-reproducible and prone to issues.

Suggested Ansible Tasks:

- name: Remove existing conda environment {{ environment_path | basename | splitext | first }}
  become: true
  ansible.builtin.command:
    cmd: "{{ miniforge_home }}/bin/mamba env remove --name {{ environment_path | basename | splitext | first }}"
  ignore_errors: true

- name: Install conda environment {{ environment_path | basename }}
  become: true
  ansible.builtin.command:
    cmd: "{{ miniforge_home }}/bin/mamba env create -f /opt/conda-environments/{{ environment_path | basename }} --prefix {{ miniforge_home }}/envs/{{ environment_path | basename | splitext | first }}"

Value and/or benefit

Implementing this solution will prevent the accumulation of outdated packages and ensure that environments remain clean, reproducible, and free from legacy issues.

Anything else?

No response

[viniciusdc]

This will also need to be removed as its now legacy https://github.com/nebari-dev/nebari-slurm/blob/main/roles/jupyterhub/templates/jupyterhub_config.py#L233

[viniciusdc]

Also, this would be something nice to be tested, in order to make sure the environments are being properly being re-created