[manualpipe] Asklepian 20220215

BioWilko commented 2 years ago

Brief description

Based on the logfile for the day it failed due to not having the rich dependency of ocarina

Pipeline exit code: 1

Executed by: @BioWilko BIRM

[x] I am sure this pipeline is not already running (Only relevant for non-flocked pipelines)

[x] Engage pipeline via mqtt from a clean dedicated mqtt environment
[ ] Disable NXF cache destruction (only applicable for ENA BAM pipe)
[x] Check output if necessary
[x] Check mqtt message is sent confirming completion, or send one if required (check in #tael-steam)
[ ] Re-enable NXF cache destruction (if disabled)

[x] Set appropriate issue flags and assignee
[x] Manual intervention complete
"Normal" service disrupted? NO
If yes, describe impact:

BioWilko commented 2 years ago

The conda environment was broken due to pip considering dependencies fulfilled by the base environment as fulfilled within the pipeline environment. @SamStudio8 fixed by re-initialising the conda environment from a VM without anything within its base environment.

SamStudio8 commented 2 years ago

Seems related to https://github.com/conda/conda/issues/448, I think we can avoid this in future with PYTHONNOUSERSITE=True in the env where devs call conda env update.

BioWilko commented 2 years ago

We also found that having conda's channel_priority set to strict caused issues solving the environment

COG-UK / dipi-group

[manualpipe] Asklepian 20220215 #186