(asdf list all dotnet) takes a lot of time

[hensou]

Is your feature request related to a problem? Please describe.

Whenever we run asdf list all dotnet it takes a lot of seconds to return the versions

Describe the solution you'd like

I would like it be as fast as possible

Describe alternatives you've considered

Additional context

[SyedMSawaid]

It is not just asdf list all dotnet but also asdf install dotnet latest that doesn't show even what it is even working on.

[MitchellCash]

It looks like the issue is all the filtering of the available versions to only return the latest patch version from each minor version series (see here). At least I think that's the goal, but there are a few bugs if it is the case:

1.1.14
1.1.14     <-- Duplicate
2.1.202    <-- Not the latest in the 2.1.x series    
2.1.818
2.2.207
3.0.103
3.1.426
5.0.408
6.0.414
7.0.401
8.0.100-rc.1.23455.8

If you revert the list-all command to go back to listing all available versions (using this function), we see a major performance improvement.

I also wonder if listing ALL available versions should be the default behaviour anyway as it matches all the other asdf plugins I use.

[hensou]

@MitchellCash thanks for your input in this. We could use the branch information, but that would not eliminate the need for the requests for the releases.json file.

For example, by using the repo tags we would have the version 8.0.0-preview.1, however, the real (downloadable) version in this case should be 8.0.100-preview.1.23115.2.

I'm still studying what are the options here, I also tested with implementing this part with perl but not sure if that makes sense. Next week I'll finally have some free time to work on this.

[MitchellCash]

@hensou ahh I understand the problem better now, thanks for explaining.

Do you know why we pipe the releases.json URL to curl just to get the latest-sdk version when that same version is available from https://raw.githubusercontent.com/dotnet/core/master/release-notes/releases-index.json directly without doing another HTTP request from https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/<VERSION>/releases.json.

For example these produce the same results:

Current behaviour:

echo $(download | match_key $FILE | sanitize $FILE | xargs curl -s | match_key $KEY | sanitize $KEY) | sed 's/ /\n/g'

Same result, but faster by avoiding the extra HTTP request for every version:

echo $(download | match_key $KEY | sanitize $KEY) | sed 's/ /\n/g'

There's probably some legacy reason why, but I couldn't figure it out.

[hydrocat]

Greetings, I tried playing around with xargs and got about 25% speedup. the main idea is to let xargs spawn more processes instead of relying on curl to sequentially download each url.

the flags that I've added are: -n 1, making each curl invocation receive a single url -P 0 makes xargs create a pool of infinite (this could be improved) sub-processes

[hensou]

This was fixed by #31 as we now have a versions.txt file with all the available versions.