Issue with File Path Autocompletion in Argparse Outside Virtual Environment

[Alfystar]

Hello,

I’m experiencing an issue with the argparse module in Python, where file path autocompletion is not functioning correctly when not using a virtual environment. The autocompletion feature, which I have enabled with argcomplete, is suggesting other optional parameters instead of file paths.

This problem occurs with a script that utilizes several nested subparsers. I’ve followed the standard procedures to enable autocompletion, including installing argcomplete, using the activate-global-python-argcomplete command, and verifying my shell configuration. Despite these efforts, the expected file path suggestions are not appearing.

Interestingly, activating a virtual environment based on Python 3.12.3 seems to resolve the issue temporarily. However, once I deactivate the virtual environment and revert to the global Python installation, the problem returns.

For a detailed description of the issue, including the structure of the subparsers and the steps I’ve taken to troubleshoot, please refer to my Stack Overflow question: Autocomplete Fails for File Paths in Argparse Without Virtual Environment.

Any insights or assistance with this matter would be greatly appreciated.

Thank you!

[evanunderscore]

Are you able to whittle your example down to the minimum code required to trigger the issue you're seeing? Once you've done that, you should be able to export _ARC_DEBUG=1 to get debug output and see where execution diverges between a working configuration and a broken one.

[Alfystar]

Hi @evanunderscore

This is the minimum code that show the problem:

#!/usr/bin/env python3
# PYTHON_ARGCOMPLETE_OK

import argparse
import argcomplete

def main():
    # Main parser definition
    parser = argparse.ArgumentParser(description="Example of argparse with subcommands.")
    parser.add_argument("--path", type=str, help="A single file path.")

    # Enabling autocompletion
    argcomplete.autocomplete(parser)
    # Parsing known arguments
    args, unknown = parser.parse_known_args()

This is my current directory:

12:03:00 ~/Documents/00_My/argvParseComplete
└─▶ $ tree
.
├── Cartella_1
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_2
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_3
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_4
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_5
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── argvParse_example.py
└── createEnvTest.sh

6 directories, 17 files

Output outside virtual Env

And this the output with the export _ARC_DEBUG=1:

12:05:40 ~/Documents/00_My/argvParseComplete
└─▶ $ ./argvParse_example.py --path  <Type TAB>
word --path split, lexer state: ' '
In trailing whitespace

LINE: './argvParse_example.py --path ' 
POINT: 30 
PREQUOTE: '' 
PREFIX: '' 
SUFFIX: '' 
WORDS: ['./argvParse_example.py', '--path']
Active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
Visited positionals: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
invoking parser with ['--path']

exception <class 'ValueError'> too many values to unpack (expected 3) while parsing args
all active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
active_parser: MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
optional options: ['-h', '--help', '--path']
next_positional: None
Active actions (L=0): []
active options: ['-h', '--help', '--path']
display completions: {'-h': 'show this help message and exit', '--help': 'show this help message and exit', '--path': 'A single file path.'}

Returning completions: ['-h:show this help message and exit', '--help:show this help message and exit', '--path:A single file path.']
-

Output inside virtual Env

12:07:39 ~/Documents/00_My/argvParseComplete
└─▶ $ source .venv/bin/activate
(.venv) 12:07:47 ~/Documents/00_My/argvParseComplete
└─▶ $ ./argvParse_example.py --path -
word --path split, lexer state: ' '
In trailing whitespace

LINE: './argvParse_example.py --path ' 
POINT: 30 
PREQUOTE: '' 
PREFIX: '' 
SUFFIX: '' 
WORDS: ['./argvParse_example.py', '--path']
Active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
Visited positionals: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
invoking parser with ['--path']

exception <class 'SystemExit'> 2 while parsing args
all active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
active_parser: MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
optional options: ['-h', '--help', '--path']
next_positional: None
Active actions (L=1): [IntrospectAction(option_strings=['--path'], dest='path', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=True, help='A single file path.', metavar=None)]
Resetting completions because IntrospectAction(option_strings=['--path'], dest='path', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=True, help='A single file path.', metavar=None) must consume the next argument
Activating completion for IntrospectAction(option_strings=['--path'], dest='path', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=True, help='A single file path.', metavar=None) <class 'argparse._StoreAction'>
Completions: ['createEnvTest.sh', 'argvParse_example.py', 'Cartella_1/', 'Cartella_2/', 'Cartella_5/', '.venv/', 'Cartella_4/', 'Cartella_3/', '.idea/']
active options: ['createEnvTest.sh', 'argvParse_example.py', 'Cartella_1/', 'Cartella_2/', 'Cartella_5/', '.venv/', 'Cartella_4/', 'Cartella_3/', '.idea/']
display completions: {'createEnvTest.sh': '', 'argvParse_example.py': '', 'Cartella_1/': '', 'Cartella_2/': '', 'Cartella_5/': '', '.venv/': '', 'Cartella_4/': '', 'Cartella_3/': '', '.idea/': ''}

Returning completions: ['createEnvTest.sh:', 'argvParse_example.py:', 'Cartella_1/:', 'Cartella_2/:', 'Cartella_5/:', '.venv/:', 'Cartella_4/:', 'Cartella_3/:', '.idea/:']
      ./argvParse_example.py --path  
usage: argvParse_example.py [-h] --path PATH
argvParse_example.py: error: argument --path: expected one argumentartella_1/  .venv/  --    

Hope this data can help you! I'm here if you need

[evanunderscore]

Which versions of Python are you using inside and outside the virtual environment? At a glance this appears to be related to #481.

[Alfystar]

Boot (env and global) are with Python 3.12.3 And i'm on Mac using zsh and bash, if can help