Community RESTful Interface standards

[LeeKamentsky]

As good a place as any to put this... We are getting some pressure (in a good way) to come up with RESTful interfaces for CellProfiler and it might be a good idea to get community consensus for what a RESTful interface to an imaging application should look like. The knime-bridge interface (https://github.com/CellProfiler/knime-bridge) was a good prototyping exercise for this - it made it clearer what's needed: introspection into inputs and outputs, given configuration parameters for the service (in the case of CellProfiler, configuration = a pipeline), standardized ways to retrieve and store images, ROIs, labelings and measurements and the metadata that describes their particulars, mechanisms to modify configurations and understand what's modifiable.

So before we all start spewing interface implementations, perhaps we should come up with the interfaces themselves? Not looking to be didactic, but pragmatic and also to take baby steps to get some useful results from a basic protocol designed so it can grow.

[ctrueden]

Great idea to discuss an effort like this. If what we're looking to define is a RESTful API for typed modules, we should perhaps put that in a new scijava/scijava-server project, and then build other RESTful APIs (e.g., for ImageJ, living here) on top of that. I am far from expert on web services, WSDL, and cross-language interface definitions. But it seems like that's what we want to start building here, no?

Anyway, I completely agree about defining some small interfaces before going hog-wild. As usual, let's start small then grow as needed. What sorts of functionality does the community initially need for CellProfiler, Lee?

[ctrueden]

I guess your list above is a good start in broad strokes—how do you want to proceed? Maybe the first step is to settle on the underlying technology framework that we'll use to code generate these things?

[LeeKamentsky]

Community needs from CellProfiler web services What Knime needs:

• Given a pipeline, what are the names of the images it needs to run, what measurements will it produce?
• Given a pipeline and images, run the pipeline and return the measurements
• Given a pipeline and an image stack, run the pipeline on the image stack as a group and return the measurements

What others need:

• Pipeline editing tools for programmatic modification of a pipeline
• RDF and ontology for measurements (from Heidelberg conference on bioimaging standards + other sources)
• Interchange of labelings, masks and upload of images.

As for coding, I think your tack is a good one for Java (javax.ws.rs). We have a lot of web services here at the Broad and they are mostly JSON instead of XML. I think the world is headed that way (see https://api.github.com). I'm not sure how self-documenting javax.ws.rs is - if the webserver serves documentation on the APIs it supports, then a reference implementation would be the API's own documentation and I like that idea.

We are trying to hire someone who has strengths in this area and possibly enlisting other people here to advise us - I will probably be able to do something further in a couple of weeks and may have someone who is more experienced to help.

IMHO - first web service should be to perform an image processing task with inputs and outputs:

• Typed inputs and outputs * Flexible typing system for later extensibility * First cut - only images as inputs and outputs
• Decide on wire format for images and whether images should be sent in-line or referenced by URL

[axtimwalde]

Hi Lee,

I like the concepts of versioning and block-access as in DVID

https://github.com/janelia-flyem/dvid

http://emdata.janelia.org/

http://emdata.janelia.org/api/help

however, it should be n-dimensional and block-access should be arbitrary. ROI-access by mask?

Best, Stephan

On Fri, 2015-04-10 at 08:42 -0700, LeeKamentsky wrote:

As good a place as any to put this... We are getting some pressure (in a good way) to come up with RESTful interfaces for CellProfiler and it might be a good idea to get community consensus for what a RESTful interface to an imaging application should look like. The knime-bridge interface (https://github.com/CellProfiler/knime-bridge) was a good prototyping exercise for this - it made it clearer what's needed: introspection into inputs and outputs, given configuration parameters for the service (in the case of CellProfiler, configuration = a pipeline), standardized ways to retrieve and store images, ROIs, labelings and measurements and the metadata that describes their particulars, mechanisms to modify configurations and understand what's modifiable.

So before we all start spewing interface implementations, perhaps we should come up with the interfaces themselves? Not looking to be didactic, but pragmatic and also to take baby steps to get some useful results from a basic protocol designed so it can grow.

---

Reply to this email directly or view it on GitHub: https://github.com/imagej/imagej-server/issues/1

[axtimwalde]

We've made great experience with Jetty

http://eclipse.org/jetty/

and GSON

https://code.google.com/p/google-gson/

in the Java world.

Best, Stephan

On Fri, 2015-04-10 at 10:22 -0700, LeeKamentsky wrote:

Community needs from CellProfiler web services What Knime needs:

• Given a pipeline, what are the names of the images it needs to run, what measurements will it produce?
• Given a pipeline and images, run the pipeline and return the measurements
• Given a pipeline and an image stack, run the pipeline on the image stack as a group and return the measurements

What others need:

• Pipeline editing tools for programmatic modification of a pipeline
• RDF and ontology for measurements (from Heidelberg conference on bioimaging standards + other sources)
• Interchange of labelings, masks and upload of images.

As for coding, I think your tack is a good one for Java (javax.ws.rs). We have a lot of web services here at the Broad and they are mostly JSON instead of XML. I think the world is headed that way (see https://api.github.com). I'm not sure how self-documenting javax.ws.rs is - if the webserver serves documentation on the APIs it supports, then a reference implementation would be the API's own documentation and I like that idea.

We are trying to hire someone who has strengths in this area and possibly enlisting other people here to advise us - I will probably be able to do something further in a couple of weeks and may have someone who is more experienced to help.

IMHO - first web service should be to perform an image processing task with inputs and outputs:

• Typed inputs and outputs * Flexible typing system for later extensibility * First cut - only images as inputs and outputs
• Decide on wire format for images and whether images should be sent in-line or referenced by URL

---

Reply to this email directly or view it on GitHub: https://github.com/imagej/imagej-server/issues/1#issuecomment-91627333

[zyme]

Hi Folks, Just wanted to introduce myself and say that I hope to contribute to some REST-ful and S3-ification. I'm a big fan of Jetty, in particular the dropwizard "stack" (http://www.dropwizard.io/), which is a great environment for writing java REST services without all the hassle and bloat of typical J2EE. Great testing tools right out of the box, and with the swagger module, it's super easy to make great online documentation of the web services with just a few annotations on your resource methods.

I like Lee's idea of figuring out the basic interfaces here first. Maybe just a handful of fairly coarse web services (the basic inputs and outputs) to start with from a conceptual level and then dive into the particulars of PUTs, GETs, authn and authz, json/xml etc?

[ctrueden]

@zyme Any traction yet on your end in these directions? The LOCI ImageJ team has had no time yet to invest in web services development. But it is still an area of interest.

[ctrueden]

After thinking some about it this morning, I think all we would need initially is:

1. A service for GETting objects in the system (notably images, but really anything available from SciJava's ObjectService).
2. A service for POSTing and/or PUTting objects into the system (again: notably images).
3. Services for GETting information about available modules.
4. A service for executing a module (in ImageJ's case: SciJava modules), with typed inputs and outputs. In ImageJ, such modules commonly take the form of Commands (i.e., plugins written in Java), and scripts, written as text in any of the supported SciJava scripting languages.
5. Optionally, a service for DELETEing objects. (Would be easy to implement.)

With just the first four things, it would be possible for interested tools like CellProfiler to:

• Send relevant inputs to ImageJ
• Execute operations on those inputs
• Extract outputs from ImageJ

And do so in a flexible enough way to avoid unnecessary trips across the language bridge. You would just need a lightweight/thin CP module to wrap each service, as well as a module for spinning up and/or shutting down the ImageJ server. The SCIFIO library would allow ImageJ to accept and produce images in a variety of formats, which is super convenient for callers.

As an aside, I think it would be straightforward for the server to support both HTTP and pipes—might be worth a quick benchmark in both directions to see whether there is a notable performance difference.

The idea would be to use JSON for simple input/output values, with references for objects (esp. images, tables, and ROIs) that persist on the server side in the object index.

We would need to adopt or define some efficient binary standard(s) for ROIs, as well as for tables probably. I don't have many concrete thoughts on this yet.

Regarding security, generally speaking, I think the server would be as secure as the modules you expose. Main vulnerability would be from super-general modules which allow things like "execute such-and-such string input or such-and-such file as a script"—this would obviously let the caller make ImageJ do whatever it wanted.

Other things the server could do later:

• Support multiple simultaneous ImageJ contexts, if you want to spin up a single server and perform various operations isolated from one another.
• Support for client-side images with pixels read on demand. This would require an inversion of the client-server relationship, with the server establishing a connection to the client for on-demand access to client-side pixels. But it be a huge performance gain in scenarios where ImageJ modules need access to only a subset of client-side image pixels—the client would no longer need to actually feed all image pixels to ImageJ, but rather only feed it an active socket. I guess we could accomplish this by simply implementing the RESTful services in reverse on the CellProfiler side eventually?

[ctrueden]

@0x00B1 (and @zyme!) pointed me at the excellent Dropwizard package for RESTful web services. So development of an ImageJ-driven RESTful image server prototype is proceeding on the dropwizard branch here.

[ctrueden]

I have not had time to continue working on the dropwizard-based implementation of imagej-server yet. But it is still very much on my radar. I am busy through July, but later this summer/fall I will definitely pursue it further.

[ctrueden]

Thanks to the efforts of @lnyng, we have merged an initial working Dropwizard-based prototype to master. There is still much work to be done; see the README for a list of some questions and issues. In particular, we need to write a formal implementation-agnostic specification and ensure that other software systems can back it the same as ImageJ here.

Feedback warmly welcomed.

[ctrueden]

The imagej-server has seen a couple of releases now. I am definitely still interested in trying to standardize image server web services beyond only ImageJ, but I personally have no bandwidth currently to explore this. If anyone wants to pick up the mantle, start a conversation on https://forum.image.sc/ and I'll do what I can to help ImageJ Server inform and conform to emerging RESTful image server best practices.