Add ability to add conda-store packages

[peytondmurray]

Summary

This PR adds the ability to add conda-store packages.

Changes

• Created new function to add a list of packages to an environment; if a package already exists in the environment, no change is made to that package.

Not sure about the CI failures - they look like an unrelated build issue.

[github-actions[bot]]

:point_left: Launch a binder notebook on the branch Quansight/gator/add-conda-store-packages

[fcollonval]

Hey @peytondmurray I pushed a PR to try fixing the CI

I wanted to test this feature. and as I finally installed conda-store on my computer. I realize that installing package does not work because the panel keep fetching data and every time it does the list is refreshed and the packages selection is lost:

Log from conda-store:

conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:52] "GET /api/v1/package/?page=10&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:53] "GET /api/v1/package/?page=11&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:54] "GET /api/v1/package/?page=12&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:55] "GET /api/v1/package/?page=13&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:56] "GET /api/v1/package/?page=14&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:57] "GET /api/v1/package/?page=15&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:58] "GET /api/v1/package/?page=16&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [02/Dec/2021 09:13:59] "GET /api/v1/package/?page=17&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -

[peytondmurray]

Ha, I think you're right - I think I need to be careful when I fetch new packages, and make sure that currently selected packages remain selected during the fetch.

[peytondmurray]

@fcollonval , I think I've got it working. Sorry for the long delay, I've been busy with other projects. I've fixed the behavior of the selected packages. Here's a demo of it working:

https://user-images.githubusercontent.com/14017872/146477097-d15bb4ee-231d-4094-ae6e-e0247c89b014.mp4

[fcollonval]

Indeed the selection is persisted. So now you can look at the next problem, if you try installing packages, you are overwhelming the server with requests:

conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/package/?page=518&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:44] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -

conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/package/?page=519&size=100&distinct_on=name&distinct_on=version&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:45] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/environment/default/web/ HTTP/1.1" 200 -
conda-store-server_1  | 172.18.0.1 - - [20/Dec/2021 08:20:46] "GET /api/v1/build/1/packages/?page=1&size=100&sort_by=name HTTP/1.1" 200 -

Using the while loop with no timeout nor throttling of the request should probably be avoided. But moreover you need a way to wait for the build to finish. I don't know the REST API of conda-store. But for this server extension, for long running task, the server endpoint returns 202 code with the location endpoint to request for the task completion status. That status endpoint is then requested periodically (once per second what is already high compare to the time to carry a conda env modification).

https://github.com/mamba-org/gator/blob/master/packages/common/src/services.ts#L866

[peytondmurray]

Closing this PR as stale.