ebi-gene-expression-group / atlas-bash-util

0 stars 0 forks source link

Feature/lsf conda env option #10

Closed irisdianauy closed 2 years ago

irisdianauy commented 2 years ago

Add an option to specify the conda environment in which to run the atlas LSF job Reference: https://github.com/ebi-gene-expression-group/atlas-bash-util/issues/9

irisdianauy commented 2 years ago

Still a WIP; I am not sure if the login node (where the lsf run is submitted) sees the same conda path as the compute node (or whichever node atlas is run). Because of this, I am not sure if I am getting the correct conda path in this line: https://github.com/ebi-gene-expression-group/atlas-bash-util/pull/10/files#diff-160a42967b810af1c3fb87280ad3b1458519861d88931777e4b79e22295dba5eR30

This die message should also probably include a warning that either the command or the supplied conda env name (or both) may not have been valid. https://github.com/ebi-gene-expression-group/atlas-bash-util/pull/10/files#diff-372f980b65fe477ce80daa59c1d26e0ad6bc2fe78323afc4c69a3b2ee6571d25R110

irisdianauy commented 2 years ago

Checking the conda path using which conda as a regular user in the codon head node gives the correct path. But doing the same as user fg_atlas gives this:

$ which conda
conda ()
{ 
  if [ “$#” -lt 1 ]; then
    “$CONDA_EXE” $_CE_M $_CE_CONDA;
  else
    \local cmd=“$1”;
    shift;
    case “$cmd” in 
      activate | deactivate)
        __conda_activate “$cmd” “$@”
      ;;
      install | update | upgrade | remove | uninstall)
        CONDA_INTERNAL_OLDPATH=“${PATH}“;
        __add_sys_prefix_to_path;
        “$CONDA_EXE” $_CE_M $_CE_CONDA “$cmd” “$@“;
        \local t1=$?;
        PATH=“${CONDA_INTERNAL_OLDPATH}“;
        if [ $t1 = 0 ]; then
          __conda_reactivate;
        else
          return $t1;
        fi
      ;;
      *)
        CONDA_INTERNAL_OLDPATH=“${PATH}“;
        __add_sys_prefix_to_path;
        “$CONDA_EXE” $_CE_M $_CE_CONDA “$cmd” “$@“;
        \local t1=$?;
        PATH=“${CONDA_INTERNAL_OLDPATH}“;
        return $t1
      ;;
    esac;
  fi
}

Taking this into consideration, querying the conda path using which conda in the older commit d4f9b03b012f329b2ab8f2c53706edca0681d7e7 was replaced in e9113085997781a58355929d8763611621b6b230 by an awk parse of conda info --json.

pinin4fjords commented 2 years ago

@irisdianauy - did you test this and is it ready to go now? Feel free to remove the [WIP] and ask for permission to merge when you're ready

pinin4fjords commented 2 years ago

Feel free to squash and merge when ready