Unified Conda JavaScript API?

[lidavidm]

So @tswicegood and I have been talking about this. The Anaconda Launcher/Wakari Express/this project all (will eventually) control Conda by making requests to a webserver that interacts with Conda; it would be beneficial if all could use the same API - what should this API look like?

Also there is the issue of the API the webserver uses to control Conda. anaconda-launcher and conda-ui call conda directly; neither use conda-api and conda.api is missing functionality for both. I think it would be better if both used a unified API (such as conda-api), as then the behavior and returned data would be the same for all users.

conda-api doesn't cover all the functionality, and the command line JSON API it depends on is also incomplete (https://github.com/conda/conda/pull/795, https://github.com/conda/conda-api/pull/4 improve this). The questions here are whether the same API should be used and what that API would look like.

Using the command line conda --json API would be nice in that it would let Anaconda Launcher, when/if eventually ported to Atom Shell, skip the webserver entirely - it could interact with the conda process directly (and the JS library could transparently handle this, if the JSON from the server and from conda --json match/are very similar).

[lidavidm]

As an example of such an API: https://github.com/lidavidm/conda-js

This library is usable in both Node and in a browser. In the former it asynchronously calls conda as a subprocess, in the latter it makes AJAX requests to the server. (For development it can send console parameters to the included dev server which will execute them and send the results back.)

// if in Node
// var conda = require('./conda');

conda.Env.getEnvs().then(function(envs) {
    envs[0].linked().then(function(linked) { console.log(linked) });
    envs[0].remove('markupsafe').done(function(info) {
        console.log('Removed package');
    });
});

[continuum-puppet]

The current conda-api was created for launcher originally. It should be extended for the other use-cases that are emerging. Both the command line json API, a python program conda.api should be priorities at this point.

-Travis

On Mon, Jul 7, 2014 at 2:48 PM, David Li notifications@github.com wrote:

As an example of such an API: https://github.com/lidavidm/conda-js

This library is usable in both Node and in a browser. In the former it asynchronously calls conda as a subprocess, in the latter it makes AJAX requests to the server. (For development it can send console parameters to the included dev server which will execute them and send the results back.)

// if in Node// var conda = require('./conda'); conda.Env.getEnvs().then(function(envs) { envs[0].linked().then(function(linked) { console.log(linked) }); envs[0].remove('markupsafe').done(function(info) { console.log('Removed package'); });});

— Reply to this email directly or view it on GitHub https://github.com/ContinuumIO/conda-ui/issues/5#issuecomment-48231402.

Travis Oliphant CEO Continuum Analytics, Inc. http://www.continuum.io

[mattpap]

As an example of such an API: https://github.com/lidavidm/conda-js

@lidavidm, so what's the reason for using nodejs? I presume this has something to do with atom (whatever this is). I would highly recommend reducing our toolchain, rather than bloating it more. But maybe you can point me to relevant discussions/design docs. Make sure to remember about long term maintenance of this software. Btw. the name is confusing, this should be conda-node or something.

[lidavidm]

@mattpap Atom Shell is Webkit packaged so that a web app can be run in its own window as a desktop app, like what we use QtWebkit for, and with additional desktop and Node.js APIs.

The reasoning behind this library is that we (will) have three projects, Anaconda Launcher, Conda UI, and Wakari Express, structured as webapps that control conda by making requests to a server. Instead of duplicating this effort, we can use a single JavaScript interface. It supports running under node because of Atom Shell, but is also designed for the browser (thus conda-js, not conda-node).

The reasoning behind Atom Shell is that it is more up-to-date/(hopefully) less buggy than what Qt provides, and because it (for Anaconda Launcher) allows us to skip the webserver entirely; conda-js will transparently control conda by running it as a subprocess instead.

[lidavidm]

Additionally @joelhullcio and @tswicegood have discussed using Atom Shell for what will eventually be Wakari Express, perhaps they could offer you more insight on that?

[tswicegood]

@mattpap, @lidavidm has been doing some exploratory work with atom-shell to see if it might work as a good alternative to using QtWeb. One of the bonuses we get in atom-shell is the ability to launch sub-processes directly via an embedded Node.js setup, thus the need for a Conda JavaScript API that can execute against a remote server and one that can execute in Node.

That's where conda-js comes in. It's meant to be that shim that provides access to conda's API either via a RESTful web service or a sub-process.

[continuum-puppet]

Is conda-js the node-js equivalent to conda-api?

-Travis

On Tue, Jul 8, 2014 at 5:46 PM, Travis Swicegood notifications@github.com wrote:

@mattpap https://github.com/mattpap, @lidavidm https://github.com/lidavidm has been doing some exploratory work with atom-shell https://github.com/atom/atom-shell to see if it might work as a good alternative to using QtWeb. One of the bonuses we get in atom-shell is the ability to launch sub-processes directly via an embedded Node.js setup, thus the need for a Conda JavaScript API that can execute against a remote server and one that can execute in Node.

That's where conda-js comes in. It's meant to be that shim that provides access to conda's API either via a RESTful web service or a sub-process.

— Reply to this email directly or view it on GitHub https://github.com/ContinuumIO/conda-ui/issues/5#issuecomment-48403656.

Travis Oliphant CEO Continuum Analytics, Inc. http://www.continuum.io

[tswicegood]

Is conda-js the node-js equivalent to conda-api?

conda-js is the browser & node equivalent of conda-api. The browser version needs a RESTful server on the machine where the conda environment is installed; the node version simply needs the conda command line tool.

FYI, @teoliphant, you're emails are coming from an address associated with continuum-puppet and not your user.

[mattpap]

I just wonder if we can make it really RESTful, i.e. stateless. For example, right now I ask for a plan and then ask to install (which recomputes the plan). I could send the entire plan back to server, assuming nothing changed in between. I also need progress bar support which implies websockets, unless we want to do long polling/comet.

In general I'm in favor of unification and simplification of our stack, so if this helps, lets proceed.

[mattpap]

@lidavidm, speaking about the API itself, you shouldn't limit your work to what conda's CLI provides. It's limited and disorganized. It looks like you thought about the later issue. Regarding the former, look e.g. at https://github.com/ContinuumIO/conda-ui/blob/master/conda_ui/api.py#L61. This wraps conda's Package class to do more than the original docstring says. Hopefully, those are all fields that can happen. If anyone can point me to documentation of those fields, it would be great. Otherwise, we should document this.

[lidavidm]

@mattpap I noticed the same and was wondering how best to handle that. Perhaps that class should be merged into conda?

[tswicegood]

@mattpap regarding conda_ui.api -- I think most of that should end up in some combination of conda-api and conda via command line options. In theory, everything in conda via conda-api should be accessible via conda the command line tool and vice versa.

@ilanschnell thoughts?

[mattpap]

Perhaps that class should be merged into conda?

I presume yes. I actually started from modifying conda, but I wasn't sure how much time it will take to review and merge those changes, and people are more likely to try out new code if setup is limited to minimum (e.g. it doesn't depend on modified versions of other packages). I actually have an obsolete branch with those changes and addition of --json to a few CLI commands, but that's unfinished and it looks like you've taken care of this issue.

[mattpap]

I think most of that should end up in some combination of conda-api and conda via command line options

It all depends if we can proved required functionality this way. If so, I have no issue with moving this code elsewhere, leaving conda-ui pure UI project. EDIT: Which I hoped it would be from the very beginning, but at that time I didn't know about that state of art of conda's APIs.

[asmeurer]

It's better to not use the Python API to conda. It will force whatever uses it to be installed in the root environment.