Deploying as a web service

[johnzeringue]

Per @bowring in #27:

On a related topic, I would also like to explore deploying Calamari as a web service and would value your insights in that regard. One of the problems for example is that Geosciences Australia's official machines are currently stuck in Java 6 because of a dependence on some Oracle products and providing Calamari as a service would bypass this issue.

[johnzeringue]

This sounds pretty easy to me. A simple idea for the API would be to have an endpoint, say POST /calamari/raw-data that takes Prawn XML and returns a list of report URLs. Whether or not we need permissions is something that could complicate matters, though not tremendously.

As I believe you noted in an email, the reports generation API needs some work. An obvious defect at the moment is that it doesn't programmatically disclose the locations of reports. This is easily fixed but appears to be the first step.

Another chore is to break the project into core and app subprojects in order to allow the addition of web. At this point, I'd like to replace Maven with Gradle, since the I find the latter easier to work with when automating deployment. #22 was just an attempt to get tests running on Travis, but with a web component I believe switching to Gradle creates value.

Jersey would be a suitable framework for the job, and Jetty's a good web server that's easy to maintain and deploy. Once the current codebase is prepared, the server is only a couple hundred lines of code.

@bowring, is there anything I've overlooked? If we can come up with a decent list of requirements, I might have time to start on it this week.

[bowring]

@johnzeringue - I need to explain the lay of the land that we have so far. Calamari has two current tasks: task one is to produce these reports that are formatted so that they can be opened in Excel and compared with similar tables produced by Squid 2.5 and give users of Squid 2.5 confidence that we are doing things correctly in terms of replication; task two is to produce a collection of ShrimpFraction objects that encapsulate the data and data reduction done by Calamari for use in other programs - i.e., ET_Redux, which currently uses some of this information as part of a demo of what we can do.

The basic idea has been to replicate the mathematical and graphical behavior of Squid using Calamari and ET_Redux - we are are only at the beginning and reluctant to commit too much more time without some feedback - we have created all this as a sales pitch to the Shrimp community. If they decide this fall to move ahead and fund us, I am sure we will be inundated with requirements. However, lowering the entry bar for using Calamari by providing a web service will be a good addition to the sales effort, even if it later undergoes changes.

So, requirements are simple at this point (work in process - please edit, organize as you see fit): 1) A free or almost free (short-term = 6 months?) server in the cloud that we control and that has no permission problems. 2) task one will become a web service, task two will remain a library; note that the API will be changing soon to accommodate flags and parameters controlling data reduction and report creation. 3) reports produced as URIs or reports as a downloaded zip file 4) a simple graphical web interface where user chooses file, sets flags and parameters, pushes a button, and gets the reports 5) Prawn files can be local or URIs (as now supported in Calamari) 6) ...

[johnzeringue]

1. I can probably provide hosting in the short term.
2. Okay. Let's get even simpler. API is POST calamari/raw-data, takes Prawn XML, and returns the reports as a ZIP. Flags and parameters will be URL-encoded. Library and web service should be able to evolve together without issue.
3. Addressed in 2.
4. This is trivial after we have our API.
5. Can we start with local first? URIs shouldn't be hard to add, but let's start with the MVP.

[bowring]

1. I am checking into the status of the promised CofC academic servers for our use as researchers.

2 thru 5. Good.