Auto-complete dependencies for project.json

[shirhatti]

[shirhatti]

Changes are live in autocomplete Still ironing out issues. Will push to release soon

[shirhatti]

Pushed initial to release.

Still has few issues with an extra quote. Investigating further http://stackoverflow.com/questions/25252517/how-to-get-rid-of-trailing-quotation-mark-in-sublime-completions

[sayedihashimi]

What's the status of this feature? What work remains so that we can complete & merge?

[shirhatti]

• Right now completion are pulled from MyGet only.

  Ideally we want to specify providers in a XML/JSON files

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="AspNetVNext" value="https://www.myget.org/F/aspnetvnext/" />
    <add key="NuGet.org" value="https://nuget.org/api/v2/" />
  </packageSources>
  <packageSourceCredentials>
    <AspNetVNext>
      <add key="Username" value="aspnetreadonly" />
      <add key="ClearTextPassword" value="4d8a2d9c-7b80-4162-9978-47e918c9658c" />
    </AspNetVNext>
  </packageSourceCredentials>
</configuration>

• Naive implementation of caching. Cache is rebuilt every time Sublime reloads. Also, completions are limited to the top 100.

Ideally we want package completions cache to persist on file system. Going forward, we can implement an API that this project consumes that aggregates completions from all sources. This should help reduce network utilization.

• Schema-based completions. Currently, json completions are limited to root-level attributes. The completions are also hard-coded.

Ideally we want to retrieve the schema from http://schemastore.org/ and offer schema-based completions

• There is also bug currently that introduces an extra quotation mark. Watch this gif http://gfycat.com/RegularCompleteCobra

[sayedihashimi]

Are these features stable enough to be merged to master/release? If so I'd recommend we do that and then open individual issues for items above. If they are not currently stable them can you help identify the minimal changes to get this merged? Thoughts?

[sayedihashimi]

Completions being limited to 100 is a big limitation here. Many packages begin with Microsoft which it seems don't show up in the suggestions. It looks like that limitation applies to version number as well. I tried to get suggestions for EntityFramework and that worked but version num suggestions for Microsoft.AspNet.StaticFiles did not work. It would be better to search the remote if no value found in the cache.

Supporting multiple package feeds is nice, but we can do that later. For now let's just hard code to my get for suggestions to get the best results/broadest suggestions. Later we will switch that to nuget.org when the time is right.

@jeffhandley @glennc Regarding suggestions for package name in project.json. We should find a way that we can include all NuGet package names in the suggestions. Perhaps we create an index for all NuGet package names and then use NuGet APIs to download newer packages (not sure if NuGet APIs enable this). Maybe we can do some of this work in the background if needed.

Suggested changes before initial release

• [ ] Figure out a way to get all package names included in suggestions. Having autocomplete being accurate is really important.
• [ ] Update version number suggestions to be dynamic if no value found in the cache
• [ ] Update static suggestions based on latest format for project.json

[shirhatti]

The reason I limited to 100 is because for queries larger than that, the MyGet API paginates the response. To cache more packages I would need more HTTP requests on Sublime init and I'm not sure about the best practices to do this in Sublime. I observed unresponsiveness in trying to deal with paginated results spanning multiple HTTP requests

Maybe we could ship a flatfile containing the possible packages and just push updates through Sublime Package Control periodically? This mechanism also deals with the version number issue.

[shirhatti]

Also, taking a step back we need to evaluate if there is a better way to provide completions on a JSON file. Currently, I perform a JSON parse everytime a completions list is triggered. This JSON parse is also not optimized as I currently had to write my own parser as there was no parser that dealt with malformed JSON as is often the case in the editor while typing.

We should ask for help, or explore to see if other projects have found a more elegant solution to this problem

[sayedihashimi]

Thanks for the details. From what I understand the extension downloads info for the top 100 packages and caches that periodically. This impacts suggestions for dependency name and versions number suggestions as well.

Here is what I'll propose as an alternative. We should have a file which contains the names of all the known NuGet packages. Then if possible we query nuget.org for the names of new packages since the last time the file was updated (the value is written in the file). According to nuget.org there are ~26,000 packages today so this list is not an insane number.

That file will drive suggestions for dependency names. The extension will update the file in the cache itself (or can write to another file and we merge content). When it makes the remote call to get updated/new package names that query should return the minimal amount of data (to get the fastest response). On every release we will make sure that file is up to date with all the latest package names.

The file should ideally be .json. If xml parsers (or some other format) makes it significantly faster to read the file we should consider other options as well.

When a user invokes suggestions when inside of a version number string we make a remote call to get the last 10 or 20 version numbers for that package. That query should just return the package version (minimal amount of data)

Regarding the json parser. Let's see how it goes. I don't think perf is that important here (these files are typically pretty small so reading them should be super fast anyways). If there are issues in how the file is being read that causes bugs then we should address those and consider other options. I like your depth approach in the reader and I think going from that to getting completions dynamically from a json schema file are do-able.

Some questions

• [ ] Is there a nuget.org api that we can call to return names of new packages? Can we filter by date modified? If not, is there a query that we can use for new package versions published after a certain date? We could also consider creating an api that is designed for these plugins. One that caches content and returns info quickly.
• [ ] If we were to call nuget.org when inside of a version string do you think the perf is going to be useable or too slow? If that is too slow then we should consider adding more info into the cache file. Having the full set of package names is important. So if a user publishes a new package on nuget.org I'd like for that to show up quickly in sublime.

[sayedihashimi]

Kulture is moving to a new home at https://github.com/OmniSharp/Kulture.

If we need to continue this conversation let's have it there. Closing this now.