RESTful API

[jmchilton]

Not sure if this is the best place to do it, but lets iterate on a RESTful API for how to fetch things to be installed. There was some desire to prefix these with something, any idea what that should be?

<installable_type> is one of galaxy_tool, galaxy_workflow, cwl_tool, cwl_workflow, galaxy_viz

• TODO: Tool shed search API.
• /api/suites List of suites. *
• /api/suites/<suite_id> Get suite - (just identifiers to installables right?)
• /api/<installable_type>s List installables *
• /api/<installable_type>s/<namespaced_id> Get versions lists for installable [versions will just be 0,1,2... for viz and workflows]
• /api/<installable_type>s/<namespaced_id>/versions/<version>[@revision] Get metadata for version of installable.
• /api/<installable_type>s/<namespaced_id>/versions/<version>[@revision]/download Get tarball for installable.

[hexylena]

This is a fine place to do it :)

• How about /api/galaxy/v1/...?
• yep to /api/suites/suite_id listing a bunch of installable identifiers (with or without versions?)

so

• /api/{visualization,tool,workflow}s/galaxyproject.iuc.seqtk ?
• /api/{visualization,tool,workflow}s/galaxyproject.iuc.seqtk/versions for just the version list?

What motivates the [@revision] notation? In #1 it was mentioned "Tools need to have strictly increasing versioning.", are we saying this does not hold true for the others? or ... ?

[jmchilton]

• /api/galaxy/v1 is probaby okay... my hesitation is that I want this to be CWL aware and awesome from the start. I guess the CWL will have its own standard that will need to serve up so maybe this isn't really a problem. Lets go with /api/galaxy/v1 for now.
• "Tools need to have strictly increasing versioning." Yes, this gets back to our argument though - I feel like the tools should be allowed to have multiple revisions ("", "@1", "@2") per version as long as there are not breaking changes. This can be an iteration 2 concern.
• I'd vote that suites are unversioned. At least by default and for now.

[hexylena]

• ok, happy to support CWL, just let me know what I need to do for that.
• also ok, thanks for the clarification. Do we actually need to store the old versions? Will anyone ever want them? (revision really, really feels like just an extended, "hidden" version field)
• I'm -0 on unversioned suites. Either we don't track versions and people installing the suite at different times get different contents, or we're tracking an internal revision at which point why not let the user control the numbering

[hexylena]

"strictly increasing versioning" - are we requiring + enforcing semver?

• if we are:
  • comparison is easy, I plop in https://github.com/k-bx/python-semver and call it a day (what I've done)
  • however lots of tool names will need to change to pass the strictness of semver
  • that presents problems for migrating all old tools
• if we're not:
  • should we just do a lame comparison to make sure we haven't seen this specific version before?
  • the additional benefit of checking that it isn't strictly greater than anything we've seen before is that if a user maintains a tool with version 1.0 and 2.0 in the same repository, they can patch v1 and v2 to v1.1 and v2.1, and neither upload would be rejected

[jmchilton]

@erasche Here is what Galaxy uses to determine lineage ordering:

https://github.com/jmchilton/galaxy-lib/blob/shed/galaxy/tools/toolbox/lineages/stock.py https://github.com/jmchilton/galaxy-lib/blob/shed/galaxy/tools/toolbox/lineages/stock.py#L49

Not the toolshed mind you, just what happens if you load two tools with the same ID in a tool panel.

I would replicate this because some people hate semver and I don't see a good reason to enforce it. If you want to do something more complicated that I'd be open to it as long as we update Galaxy.

I don't mind allowing multiple branches of tools as long as any unversioned operation gets the maximally versioned and not latest versioned tool.

[hexylena]

Super, works for me :)

[hexylena]

@jmchilton There's a VM sitting in the cloud with our names ssh pubkeys on it (root@shed.hx42.org, if you need to muck about with anything. Django admin password is in /root/passwords.txt)

I implemented enough of the "old" TS api that planemo can upload suites to it, virtually unchanged. https://shed.hx42.org/api/drf/installables

I'll work more on the new API going forward, but it was just so easy to implement the old one and call it a day...