Deduplicate provided subcommands

[wilhelmklopp]

Is your feature request related to a problem? Please describe.

With my Django fig autocomplete spec, there are some subcommands which are available for all Django projects. These are statically present in my spec. Then there are other commands which are project specific (custom management commands). These are loaded in dynamically using generateSpec.

Currently my spec isn't smart enough to distinguish between the statically present definitions and those that are dynamically loaded. Since the code in generateSpec loads all subcommands that Django is aware of, it will include duplicate subcommands.

And that currently looks like this:

Describe the solution you'd like

It would be awesome if the logic in my spec didn't have to account for any already-loaded suggestions and if Fig could natively deduplicate any duplicate suggestions

Describe alternatives you've considered

My spec could not rely on any static subcommand definitions and try to load everything dynamically. This was my original approach, and it works (in the right circumstances where the $PATH is set and all environment variable are present), but it's pretty slow.

I'd love to instead use this hybrid approach where the common autocomplete suggestions load instantly (due to their static definition in my spec), and then the dynamic ones are added to the list of suggestions as the much slower generateSpec and executeShellCommand finish

[QuiiBz]

It would be awesome if the logic in my spec didn't have to account for any already-loaded suggestions and if Fig could natively deduplicate any duplicate suggestions

This should be fairly easy to implement. However, users should not rely on this feature - completion specs should be made in such a way they don't duplicate suggestions (when possible of course).

[wilhelmklopp]

This should be fairly easy to implement. However, users should not rely on this feature - completion specs should be made in such a way they don't duplicate suggestions (when possible of course).

Great to hear!

For what it's worth, as a completion spec author, I would love to be able to rely on this feature. I think it's always going to be more work for me to keep track of what's already suggested and then filter based on that in the logic in my spec, so being able to fully rely on Fig to take on this work would be very neat 🙌

[wilhelmklopp]

A couple more thing that feel worth mentioning:

generateSpec behavior

With respect to generateSpec, I initially assumed that Fig shows those suggestions it already has available (ie statically defined ones) immediately and then once generateSpec finishes running, adds any further suggestions to the list visible to users. But I'm not sure if this is actually how it works? In a world where some commands are always present and can be statically defined and others can only be known once generateSpec + executeShellCommand have finished, such a behaviour makes sense to me.

For example, let's say we have 2 commands: python manage.py test (which runs tests in django) python manage.py my_custom_command (an arbitrary custom command, which just like regular commands can have a description, arguments, options, etc.)

test is native to Django, so we can define it in a static Fig spec. my_custom_command is project specific, so we can't put it in the static Fig spec, we have to use generateSpec. We can find out about my_custom_command's existence by running python manage.py help, which lists all available commands (both native and custom). However the problem with this approach is that it can be pretty slow. Under the hood python manage.py help loads the whole django project so it can take a full 1-2 seconds to finish (or longer).

That leaves us with a couple of options:

1. Don't use generateSpec and only rely on static definitions. Great because we get the suggestions for python manage.py test really quickly, but bad because we can't get the suggestions for python manage.py my_custom_command This is the approach I've currently chosen for the Django spec
2. Wait for generateSpec to finish before showing any suggestions. Good because we get suggestions for all available commands, but bad because it's slow. Even suggestions for python manage.py test (which should be available instantly) are delayed until generateSpec finishes. I believe this is the approach Fig currently uses
3. Show suggestions from static definitions immediately and then add the dynamically loaded suggestions as soon as generateSpec finishes. This has the benefit of showing that which can be shown immediately without losing out on the suggestions that can only be loaded in more slowly. The potential downside is that adding suggestions as they are loaded could be confusing to some users. This is the approach I would suggest considering

deduplication/merging

For this one I was just going to add that I think there can be cases where the definitions created by generateSpec can contain additional detail.

So for example there might be a top level subcommand called abc which is both present in the static fig definition and also getting returned by generateSpec. A basic deduplication implementation might just throw away the output from generateSpec since the subcommand is already present. But I think there can be cases where the output from generateSpec contains more information about options/arguments than was present in the static definition. It would be great if that additional information can be retained.

So perhaps what I'm asking for here is less "deduplication" and more akin to "merging". So that there is one entry in the list of suggestions for the abc subcommand, which contains all the context provided by both the static definition and generateSpec

[clo4]

This behaviour would also help with the twilio CLI spec I'm working on.

Summary: The spec uses an optional argument that has a generator. The generator helps you build a (hidden) subcommand. Once you've finished building the subcommand, because the name now matches, it shows up in the suggestions list:

I could get around this by re-running the generator when the last token matches the name of a hidden subcommand to remove the generated suggestion, but that seems like an unnecessary amount of effort...

[mschrage]

Has this been implemented @fedeci?

[fedeci]

We discussed this in discord but has never been done, happy to fix it tomorrow/later today!

[fedeci]

Ref withfig/autocomplete-engine#197