[DEFECT] Allow trying again when removing environment fails

[joshua-cogliati-inl]

Thank you for the defect report

• [X] I am using the latest version of RAVEN.
• [X] I have read the Wiki.
• [ ] I have created a minimum, reproducible example that demonstrates the defect.

Defect Description

Sometimes, when a filesystem is slow, conda will fail.

Example (from https://civet.inl.gov/job/1899935/ ):

BUILD_ROOT/raven/: scripts/establish_conda_env.sh --install --conda-defs /apps/local/mambaforge/etc/profile.d/conda.sh --no-mamba
/apps/local/mambaforge/bin/mamba
... not using mamba
CONDA
... Run Options:
... ECE Mode: 2
... Verbosity: 0
... Clean: 0
... Install Mode: CONDA
... Conda Defs: /apps/local/mambaforge/etc/profile.d/conda.sh
... Installing RAVEN libraries ...
... Detected OS as --os linux ...
... Using Python command python
... RAVEN library name set through $RAVEN_LIBS_NAME global variable.
... >> If this is not desired, then unset the variable before running.
... >> RAVEN environment is named "raven_libraries"
... Found conda definitions at /apps/local/mambaforge/etc/profile.d/conda.sh
conda 23.3.1
raven_libraries_lemhi     /home/civet/.conda/envs/raven_libraries_lemhi
raven_libraries_sawtooth     /home/civet/.conda/envs/raven_libraries_sawtooth
raven_libraries_tf       /home/civet/.conda/envs/raven_libraries_tf
                         /home/civet/miniconda3/envs/raven_libraries
                         /home/civet/miniconda3/envs/raven_libraries_dask
... Found library environment ...
... Removing old environment ...

DirectoryNotACondaEnvironmentError: The target directory exists, but it is not a conda environment.
Use 'conda create' to convert the directory to a conda environment.
  target directory: /home/civet/.conda/envs/raven_libraries

Steps to Reproduce

Run install again on sawtooth.

Expected Behavior

The install would just try again.

(As in, if this is repeated, it usually works.)

Screenshots and Input Files

No response

OS

Linux

OS Version

No response

Dependency Manager

CONDA

For Change Control Board: Issue Review

• [x] Is it tagged with a type: defect or task?
• [x] Is it tagged with a priority: critical, normal or minor?
• [x] If it will impact requirements or requirements tests, is it tagged with requirements?
• [x] If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
• [x] Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure

• [x] If the issue is a defect, is the defect fixed?
• [x] If the issue is a defect, is the defect tested for in the regression test system? (If not explain why not.)
• [x] If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
• [x] If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
• [x] If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?