Completing a CLI with `<mask>... <file>` never offers `<file>` completions with Rust native completions

[windsource]

Please complete the following tasks

• [X] I have searched the discussions
• [X] I have searched the open and rejected issues

Rust Version

rustc 1.80.1 (3f5fd8dd4 2024-08-06)

Clap Version

clap 4.5.16, clap_complete 4.5.23

Minimal reproducible code

use std::ffi::OsStr;

use clap::{CommandFactory, Parser, ValueHint};
use clap_complete::{ArgValueCompleter, CompleteEnv, CompletionCandidate};

fn mask_completer(_: &OsStr) -> Vec<CompletionCandidate> {
    vec![
        CompletionCandidate::new("foo"),
        CompletionCandidate::new("baz"),
    ]
}

#[derive(Parser, Debug)]
#[command(name = "cmdtest")]
#[command(version, about, long_about = None)]
struct Args {
    #[arg(required = true, add = ArgValueCompleter::new(mask_completer))]
    mask: Vec<String>,

    #[arg(required = true, value_hint = ValueHint::FilePath)]
    file: String,
}

fn main() {
    CompleteEnv::with_factory(Args::command).complete();

    let args = Args::parse();

    for m in args.mask {
        println!("Mask: {}", m);
    }
    println!("File: {}", args.file)
}

Steps to reproduce the bug with the above code

cmdtest foo AND PRESSING TAB

Actual Behaviour

As completion candidates I only get "foo" and "baz"

Expected Behaviour

I was expecting to "foo", "baz" and the files in the current folder as completion candidates.

Additional Context

After the first argument for mask is passed, the next argument could either be a mask or a file so I was expecting to get completions for both but I only get completions for mask.

I am using zsh.

Debug Output

No response

[epage]

This is parsing two positionals in a row with the first positional's length being unbounded. This is a very complicated case in the parser that we've not covered yet with the new completions. Currently, our quality bar is "is it as good as the old completions". If you happened to use the old completions before and can report back on its behavior in this scenario, that can help us prioritize this.

See https://github.com/clap-rs/clap/blob/fe810907bdba9c81b980ed340addace44cefd8ff/clap_builder/src/parser/parser.rs#L288-L358

What will be interesting about this case is that clap definitively knows when mask is done and file starts because it can peek ahead on the command-line. While the command-line is being written, we can't guarantee there will be anything to peek ahead to. Effectively, we'll need to just assume that any positional after the minimum possible is either mask or file and complete it as such.

To help in showing either completion, we should probably group completions by their argument by default (see #5651).

[windsource]

@epage not sure what you mean with "old completions"? Is there any older clap_complete release which I should try to see if the completions for my scenario work with?

[epage]

CompleteEnv is an in-work completion system that is unstable. clap_complete::aot is what we have historically provided.