bw2 / ConfigArgParse

A drop-in replacement for argparse that allows options to also be set via config files and/or environment variables.
MIT License
714 stars 121 forks source link

subcommands and `nargs > 1` #275

Open fopina opened 2 years ago

fopina commented 2 years ago

These lines: here and here

These always place env and config file variables at the end of argv, when any config used has nargs > 1.

While it does not make any difference in most cases, it does when subparsers are used.

If parent parser has one of these flags and it is used via env/configfile, it is moved to the end of argv, after the subparser, making them options of the subparser, not the parent one (and failing to parse args).

Not sure I understand the reason behind this logic so unable to propose a fix / submit PR.

I assume best way would be to rebuild argv based on all the parser definitions rather than assuming where to put the flags

tbooth commented 1 year ago

I've just discovered the same bug after 2 hours of head scratching. I was having trouble running snakemake and I tracked the problem down to this module. Here's the simplest script I could make that shows the issue:

import configargparse
import os

# Set a single env var as config
os.environ['FOO'] = "configured_value"

# The first iteration works fine. The second not so much.
for f_nargs in [None, '?']:

    p = configargparse.ArgParser()
    p.add('--foo', '-f', nargs=f_nargs, env_var='FOO', required=False)
    p.add('bar', nargs='*')

    options = p.parse_args(['--', 'arg1', 'arg2'])
    print("when f_nargs is {!r}".format(f_nargs))
    print(options)
    print("----------")

And here's what I get when I run it:

when f_nargs is None
Namespace(foo='configured_value', bar=['arg1', 'arg2'])
----------
when f_nargs is '?'
Namespace(foo=None, bar=['arg1', 'arg2', '-f=configured_value'])
----------

Not good!

fopina commented 1 year ago

I think this is one hard to tackle, at least for env.

The easiest way I worked around it was to have different env prefixes setup for the different parsers. That worked nicely and doesn't look bad.

Not sure how the library could handle it with same prefix as env won't be ordered, so when parent parser and subcommand do share a flag with same name, it's impossible to know which to map the env value.

John15321 commented 4 months ago

Shouldnt this be closed?