conda-incubator / setup-miniconda

Set up your GitHub Actions workflow with conda via miniconda
https://github.com/marketplace/actions/setup-miniconda
MIT License
405 stars 69 forks source link

Getting error on miniconda conda-incubator/setup-miniconda@v2.1.1 with no base environment activated #218

Open ajcost opened 2 years ago

ajcost commented 2 years ago

Using the below yaml:

name: QA - Precommit Checks
on: [push]
jobs:
  Precommit-Checks:
    runs-on: self-hosted
    steps:
      - run: echo "Checking branch precommit hooks ..."
      - run: echo "Now running on a ${{ runner.os }} server ..."
      - run: echo "The name of your branch is ${{ github.ref }} ..."
      - name: Check out repository code
        uses: actions/checkout@v2
      - name: Setup Miniconda
        uses: conda-incubator/setup-miniconda@v2.1.1
        with:
          activate-environment: sigma-sealevel
          environment-file: environment.yml
          python-version: 3.10
          auto-activate-base: false
      - run: |
          pre-commit run --all-files

I get a very strange error at Setup Miniconda with

Run conda-incubator/setup-miniconda@v2.1.1
Gathering Inputs...
Creating bootstrap condarc file in /home/ubuntu/.condarc...
Ensuring installer...
Error: No installed conda 'base' enviroment found at 

Even with the enviroment typo, not sure what that is about, some guidance would be appreciated!! Thank you

czyzi0 commented 2 years ago

Hi, I've encountered identical error but when I've specified particular url of installer to use with installer-url parameter it seemed to work ok. I don't know why default setting does not work but this may be a temporary workaround.

ghost commented 2 years ago

I have encountered the same issue on a self-hosted Github runner (Ubuntu Azure VM). In my case, I made sure to install Miniconda on the VM and have the env variable CONDA and PATH properly set by adding the script /etc/profile.d/set_conda_envs.sh with the following content:

export CONDA="/home/adminuser/miniconda3"
export PATH="$CONDA/condabin:$PATH"

If I execute a step where I run the commands:

- name: Check Conda local install
        run: |
          echo $PATH
          echo $CONDA
          conda -h
          conda env list

everything works as expected, but when I use conda-incubator/setup-miniconda@v2.1.1 I get the error:

Run conda-incubator/setup-miniconda@v2.1.1
Gathering Inputs...
Creating bootstrap condarc file in /home/adminuser/.condarc...
Ensuring installer...
Error: No installed conda 'base' enviroment found at 

@czyzi0 : how does your conda-incubator/setup-miniconda@v2.1.1 look like? What did you put in the installer-url parameter?

czyzi0 commented 2 years ago

My step with setup-miniconda is:

- name: Set up env
- uses: conda-incubator/setup-miniconda@v2
- with:
    installer-url: https://repo.anaconda.com/miniconda/Miniconda3-py38_4.11.0-Linux-x86_64.sh
    activate-environment: envname
    enivornment-file: environment.yml
    auto-activate-base: false

And it works fine like that.

Hope this helps.

ghost commented 2 years ago

@czyzi0 : thank you for the quick answer. I tried it in my setup and I get a weird behavior. The first time I run it, everything works as expected: conda gets installed and my script runs successfully thanks to the defined conda environment. If I run it again, the Github workflow fails. Additionally, if I try to ssh into my runner VM, the connections is immediately closed complaining that conda is not installed!

After some debugging, I found out that the installation process managed by conda-incubator/setup-miniconda defines a .profile in the user home folder with the following content:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/adminuser/actions-runner/_work/flux.passenger_prediction/flux.passenger_prediction/3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/adminuser/actions-runner/_work/.../3/etc/profile.d/conda.sh" ]; then
        . "/home/adminuser/actions-runner/_work/.../3/etc/profile.d/conda.sh"
    else
        export PATH="/home/adminuser/actions-runner/_work/.../3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

  # ----------------------------------------------------------------------------
  # Conda Setup Action: Basic configuration
  set -eo pipefail
  # Conda Setup Action: Custom activation
  conda activate "test"
  # ----------------------------------------------------------------------------

meaning that every new shell that is opened will try to activate the env test, but conda itself has been deleted at the end of the workflow and of course the command fails. If I remove that file, I can execute the workflow again (and ssh into the VM as expected).

So I will keep looking for an alternative solution 😄

goanpeca commented 2 years ago

@mist2410 could you have a default conda installation in your runner so that can be used?

Did you make any progress?

It seems like there is not much we can do besides provide documentation on these cases.

Cheers!


Any thoughts @jaimergp @bollwyvl @conda-incubator/setup-miniconda ?

ghost commented 2 years ago

@goanpeca , I tried to install miniconda on my runner, but conda-incubator/setup-miniconda fails to find it, even if I try to properly set the env variables CONDA and PATH (I tested it by printing them out in my Github workflow). I get the same error mentioned by @ajcost and me in our reports (https://github.com/conda-incubator/setup-miniconda/issues/218#issuecomment-1128675476).

My solution at the end was to completely skip the use of conda-incubator/setup-miniconda and try to replicate its functionalities with a set of shell commands in my workflow:

name: Workflow
on:
  push:

  run-pipeline:
    runs-on: ubuntu_runner

    defaults:
      run:
        shell: bash -l {0}

    steps:
      - name: Checkout repo contents
        uses: actions/checkout@v2

      - name: Setup Conda environment
        run: |
          conda env create -f conda.yml -n test
          conda env list

      - name: Execute MLFlow experiment
        run: |
          conda activate test
          python run.py

      - name: Remove Conda environment
        run: |
          conda deactivate
          conda env remove --name test
goanpeca commented 2 years ago

@mist2410 thanks for the feedback.

Dit you try https://github.com/conda-incubator/setup-miniconda#example-5-custom-installer ?

If that is still failing, could you provide some logs so we can make sure that use case still works?

Thanks a lot :)

ghost commented 2 years ago

@goanpeca , yes, I tried (see https://github.com/conda-incubator/setup-miniconda/issues/218#issuecomment-1128859867) but a slightly different custom installer than the one reported in the documentation. I followed the suggested version from @czyzi0 (see https://github.com/conda-incubator/setup-miniconda/issues/218#issuecomment-1128728941).

Should I try something different? Do you have other suggestions?

goanpeca commented 2 years ago

Than ks @micstadsb I guess the solution would be to remove the lines between # >>> conda initialize >>> when the process finished... or when it starts (or both)

casperdcl commented 2 years ago

The URL version fails when you restart a job (it tries to re-download miniconda and somehow gets confused).

This worked for me:

    - uses: conda-incubator/setup-miniconda@v2
      with:
        environment-file: ...
        activate-environment: ...
        python-version: ...
        miniconda-version: latest
        auto-activate-base: false
    - shell: bash --login {0}
      run: ...