InvalidRequirement: Expected package name at the start of dependency specifier

[moi90]

Checklist

• [X] I added a descriptive title
• [X] I searched open reports and couldn't find a duplicate

What happened?

The environment is defined as follows:

name: base
category: dev
channels:
  - conda-forge
  - defaults
dependencies:
  - make # For docs
  - pip
  - pip:
    - -r docs/requirements.txt

Including requirements.txt in this way is allowed by conda. However, with conda-lock -f environment.base.yml, I get the following error:

Traceback (most recent call last):
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 35, in __init__
    parsed = _parse_requirement(requirement_string)
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 73, in _parse_requirement
    name_token = tokenizer.expect(
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 140, in expect
    raise self.raise_syntax_error(f"Expected {expected}")
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 165, in raise_syntax_error
    raise ParserSyntaxError(
pkg_resources.extern.packaging._tokenizer.ParserSyntaxError: Expected package name at the start of dependency specifier
    -r docs/requirements.txt
    ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mambauser/.local/bin/conda-lock", line 8, in <module>
    sys.exit(main())
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1320, in lock
    lock_func(
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1033, in run_lock
    make_lock_files(
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 327, in make_lock_files
    lock_spec = make_lock_spec(
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/__init__.py", line 89, in make_lock_spec
    lock_specs = _parse_source_files(src_files, platforms)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/__init__.py", line 70, in _parse_source_files
    desired_envs.append(parse_environment_file(src_file, platforms))
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/environment_yaml.py", line 127, in parse_environment_file
    dep_map = {
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/environment_yaml.py", line 128, in <dictcomp>
    platform: _parse_environment_file_for_platform(content, category, platform)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/environment_yaml.py", line 70, in _parse_environment_file_for_platform
    parse_python_requirement(
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/pyproject_toml.py", line 409, in parse_python_requirement
    parsed_req = parse_requirement_specifier(requirement)
  File "/home/mambauser/.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/src_parser/pyproject_toml.py", line 383, in parse_requirement_specifier
    return Requirement.parse(requirement_specifier)
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3215, in parse
    (req,) = parse_requirements(s)
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3174, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/mambauser/.local/pipx/shared/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
pkg_resources.extern.packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
    -r docs/requirements.txt
    ^

Conda Info

active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /home/mambauser/.condarc
 populated config files : /home/mambauser/.condarc
          conda version : 23.5.0
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/conda/pkgs
                          /home/mambauser/.conda/pkgs
       envs directories : /opt/conda/envs
                          /home/mambauser/.conda/envs
               platform : linux-64
             user-agent : conda/23.5.0 requests/2.31.0 CPython/3.10.12 Linux/5.15.0-76-generic ubuntu/22.04.2 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

Conda Config

==> /home/mambauser/.condarc <==
channels:
  - conda-forge

Conda list

No response

Additional Context

No response

[maresb]

I'm not saying no, but I'm not convinced that we should support this.

Anything wrong with replacing that line with conda-lock -f environment.base.yaml -f docs/requirements.txt?

[moi90]

Anything wrong with replacing that line with conda-lock -f environment.base.yaml -f docs/requirements.txt?

docs/requirements.txt is only a dev requirement, so that would not quite work as intended. Or is there a way to specify a category in a requirements.txt, maybe via a comment?

[maresb]

Oh, that's very interesting! I think we're missing this feature. A comment seems like a reasonable way to provide this info. I'm not sure the best way to design the specification. Do you have any suggestions?

CC: @mariusvniekerk

[moi90]

The documentation only talks about environment.yml, meta.yml and pyproject.toml as source formats. I didn't even know that requirements.txt is also a valid input format. So this is a great opportunity to improve the documentation :)

Regarding the exact format: Maybe something like #category: dev, #!category: dev or #--category dev? Is there some de-facto standard to include metadata in comments?

[maresb]

Ah, my bad, we don't support requirements.txt yet. I was confused since almost all the code to accomplish this exists already. This should be fairly straightforward to implement. (Look under conda_lock/src_parser.) Would you be interested in submitting a PR?

Regarding the format, of your suggestions I personally prefer the first.

[moi90]

I'm currently not able to provide a PR, sorry. But it's great that so much is already in place.