Extend Pylance to include "dynamic" suggestions in the autocomplete

[alexlatchford]

The organization I work for has a well curated metadata catalog of datasets with a queryable autocomplete service. I work on a team that supports our Machine Learning teams and a common feature request we get is to improve the ability to discover data and use it experimentally to test out new modeling ideas. We're in the early stages of thinking through a design where IDE autocompletes could be used to discover datasets "dynamically".

From my limited understanding of Pylance it has an indexing phase where it'll analyze the environment is connected to, it is unclear to me how frequently that index is updated or whether it's possible to mutate that based on remote calls to external services based on the current prompt.

Some psuedo-code of the UX we're thinking might be a good fit for the user case:

mds = MetadataStore(...)
# As the user begins to type "customer" it'll query the remote system to generate a candidate set of matching dataset names.
df: pd.Dataframe = mds.datasets.custom 
# Once autocomplete row is selected it'd be: df: pd.Dataframe = mds.datasets.customers(...)

# Ideally it'd also populate the docstring of the "customers(...)" with arguments for partitioning/filtering options, 
# schema info as well for the "Customer" datatype in this case (and map it appropriately to common toolchains 
# like Pandas/Spark etc.).

Obviously is a high-risk high-reward scenario, lots of failure conditions here if there is latency/failures on the remote system but assuming we can handle those is this a scenario that would need to be implemented in Pylance or somewhere else? For reference we have scientists that use a variety of IDEs (mostly PyCharm and Vim/Emacs as well as VSCode) so likely we'll be looking for a solution that'd span that gamut. Any ideas greatly appreciated!

Apologies if this is discussed elsewhere, but I tried and failed to find anything that seemed similar!

[rchiodo]

There's actually already a way to do this in VS code. You provide your own language server for python. In fact, this is how the Jupyter extension provides 'dynamic' completions using the Jupyter kernel.

VS code documentation for language servers is here: https://code.visualstudio.com/api/language-extensions/language-server-extension-guide

[alexlatchford]

Sweet thanks Rich, let me dig into those links. I'll close out this issue and reopen if needs be!