Additional dependencies being silently ignored when using two spaces

[brynpickering]

I'm creating some actions that all start by creating a mamba environment using setup-micromamba. I have had no issues with it until this one random case that does not seem to be a purely micromamba issue.

I have one job that installs most dependencies from two requirements files, then also pins the python version and adds the ruff dependency in create-args:

jobs:
  memory-profiler:
    runs-on: ubuntu-latest
    steps:    
      - uses: actions/checkout@v3
      - uses: mamba-org/setup-micromamba@v1
        with:
          micromamba-version: latest
          environment-name: ubuntu-latest-311
          environment-file: requirements/base.txt
          create-args: >-
            -c city-modelling-lab 
            -f requirements/dev.txt
            ruff=0.0.286
            python=3.11
          post-cleanup: all
          cache-environment: true
    ...

This works fine, as can be seen in e.g. this workflow run. Notably, the ruff dependency is installed when creating the environment.

But then I have a similar job that adds in some memory profiling dependencies in create-args. This fails:

jobs:
  memory-profiler:
    runs-on: ubuntu-latest
    steps:    
      - uses: actions/checkout@v3
      - uses: mamba-org/setup-micromamba@v1
        with:
          micromamba-version: latest
          environment-name: ubuntu-latest-311-memray
          environment-file: requirements/base.txt
          create-args: >-
            -c city-modelling-lab 
            -f requirements/dev.txt
            memray=1.9.1
            pytest-memray=1.5.0
            python=3.11
          post-cleanup: all
          cache-environment: true
    ...

This second case is mysterious, because it translates to the correct micromamba call:

/home/runner/micromamba-bin/micromamba create -y -r /home/runner/micromamba -f requirements/base.txt -n ubuntu-latest-311-memray -c city-modelling-lab  -f requirements/dev.txt memray=1.9.1 pytest-memray=1.5.0 python=3.11 --log-level warning --rc-file /home/runner/micromamba-bin/.condarc

But then memray and pytest-memray are not in the full list of installed packages under micromamba list.

Running the call locally creates the expected environment, i.e. including memray.

[jonashaag]

Is it because the list is run in the first environment?

Can you link to a run that shows this problem please?

[pavelzw]

this run shows the issue @jonashaag

no memray to be seen :/

[pavelzw]

This seems like a mamba problem... @jonashaag do you know if multiple dependency files are supported by micromamba?

[pavelzw]

Running the call locally creates the expected environment, i.e. including memray.

Did you also try it out manually in CI? i.e. run

- uses: mamba-org/setup-micromamba@v1
- run: /home/runner/micromamba-bin/micromamba create -y -r /home/runner/micromamba -f requirements/base.txt -n ubuntu-latest-311-memray -c city-modelling-lab  -f requirements/dev.txt memray=1.9.1 pytest-memray=1.5.0 python=3.11 --log-level warning --rc-file /home/runner/micromamba-bin/.condarc
- run: |
    micromamba activate ubuntu-latest-311-memray
    micromamba list
  shell: bash -el {0}

[jonashaag]

Just tried locally and this works fine

... -f env1.txt -f env2.txt some-other-pkg

[jonashaag]

When I try the exact command from the workflow it works

❯ micromamba create -y -r /home/runner/micromamba -f requirements/base.txt -n ubuntu-latest-311-memray -c city-modelling-lab  -f requirements/dev.txt memray=1.9.1 pytest-memray=1.5.0 python=3.11
...

Transaction

  Prefix: /System/Volumes/Data/home/runner/micromamba/envs/ubuntu-latest-311-memray

  Updating specs:

   - memray=1.9.1
   - pytest-memray=1.5.0
   - python=3.11
   ...

Weird...

[brynpickering]

Can you link to a run that shows this problem please?

@jonashaag I linked it in the initial message, but for ease here they are again:

1. A successful run with "Ruff" as an additional dependency, but otherwise all other args the same as (2)
2. An unsuccessful run with "memray" and "pytest-memray" as additional dependencies, but otherwise all other args the same as (1)
3. Running a (successful) manual install in the CI (following @pavelzw's suggestion)

The unique thing about (2) is that the two additional dependencies are just completely ignored in the mamba install specs. Here is what it should look like (from (3)):

Here is what (2) does:

[brynpickering]

To add, I tried switching around the dependencies and removing the version pinning, but that made no difference (see here).

[jonashaag]

Thank you! I really have no clue what's going on here; we are calling micromamba with the correct arguments and it still ignores the deps. We cannot reproduce outside of the action, which is very weird.

Can you please increase log level to trace, maybe there is some additional info there.

[brynpickering]

With increased log level: here

I don't see anything in the log to suggest a problem, it's just ignoring the additional args entirely.

[jonashaag]

At this point I'm just proposing random things because I have no clue what's going on.

What if you try to install ONLY the memray stuff?

[brynpickering]

OK, I tried multiple variations and found out it was caused by a double space before the -f in the list of arguments:

❌ -f requirements/base.txt -n ubuntu-latest-311-memray -c city-modelling-lab -f requirements/dev.txt memray=1.9.1 pytest-memray=1.5.0 python=3.11

✅ -f requirements/base.txt -n ubuntu-latest-311-memray -c city-modelling-lab -f requirements/dev.txt python=3.11 memray=1.9 pytest-memray=1.5

However, this shouldn't trip micromamba up. It works fine locally with that double space and it works fine on the CI when that double space is included in a manual run.

[jonashaag]

Nice find. @pavelzw could it be that setup-micromamba passed an '' arg?

[brynpickering]

BTW, the double space was my error!

[pavelzw]

could it be that setup-micromamba passed an '' arg?

Yes, it seems so 🤔

https://github.com/mamba-org/setup-micromamba/blob/6e8c96efc14a6be3bb85fdfafd148b5e8967b429/src/options.ts#L95

I probably need to add a filter here.

[pavelzw]

I was able to reproduce it with a minimal example 👍🏻 https://github.com/mamba-org/setup-micromamba/actions/runs/6075074645/job/16480532390?pr=138

  create-args-multiple-spaces-2:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./
        with:
          environment-file: test/environment2.yml
          create-args: -c bioconda  pytest
      - run: |
          micromamba list | grep -q python
          micromamba list | grep -q pytest
        shell: bash -el {0}

[pavelzw]

Thanks for your help in figuring this out @brynpickering! ❤️

This issue should be fixed in the 1.4.4 release.