Adjust .travis.yml & build scripts to build new Akka Http API

kellyi commented 6 years ago

Overview

This PR makes a few adjustments to the project in order to have Travis build the new Akka Http jar on releases. The changes here include:

renaming api project -> mmw-geoprocessing-api
adjusting cibuild to use Docker-based ./sbt "mmw-geoprocessing-api" assembly
adjusting the .travis.yml file to:
- drop language: scala, which would cause the build job to prep for building with the Travis sbt even though we brought our own
- call cibuild script in script command
- remove Docker-based assembly from before_deploy, since it happens in cibuild now

This is preparatory work which Connects https://github.com/WikiWatershed/model-my-watershed/issues/2101

Demo

https://travis-ci.org/WikiWatershed/mmw-geoprocessing/builds/260363949

Notes

As a consequence of this Travis now runs cibuild for non-release branches of the project; previously, non-deploy branches passed CI each time due to the /bin/true call. I think it makes sense to try to build other branches, but we can revert that and call cibuild in before_deploy if we want all non-release branches to pass.

Testing Instructions

pull this branch, then run cibuild and confirm that it builds
restart the build here and verify that it also builds correctly:

https://travis-ci.org/WikiWatershed/mmw-geoprocessing/builds/260363949

kellyi commented 6 years ago

Forgot to mention that you'll have to have Docker running for cibuild to work locally now. We don't have a VM for this project -- and it's probably unnecessary -- but if we keep it this way I think it makes sense to add a note that cibuild requires Docker to the README.

rajadain commented 6 years ago

renaming api project → mmw-geoprocessing-api

Why was this required?

kellyi commented 6 years ago

Why was this required?

Wasn't required per se but since we're building only "project api" here I figured it'd be worth renaming it so that the jar won't be named api-assembly-3.0.0-alpha.jar.

Some alternatives would be: a) keep the name as "api" or b) change it to mmw-geoprocessing and change the previous project to mmw-geoprocessing-summary. Happy to do whichever we'd like.

rajadain commented 6 years ago

Oh hmm. I do want it to be mmw-geoprocessing-api-3.0.0-alpha.jar. But since previously when building the old summary project we'd get a mmw-geoprocessing-summary file, without needing to rename the project mmw-geoprocessing-summary, I guess I'm not clear why that's not happening any more. Is it because of we moved this line?

rajadain commented 6 years ago

Hrrmm I'm also getting Docker for Mac errors running cibuild now:

> kj cibuild
+ mkdir -p /Users/ttuhinanshu/.sbt
+ docker run -v /Users/ttuhinanshu/.ivy2:/root/.ivy2 -v /Users/ttuhinanshu/.sbt:/root/.sbt -v /Users/ttuhinanshu/dev/mmwg:/mmw-geoprocessing -w /mmw-geoprocessing quay.io/azavea/scala ./sbt 'project mmw-geoprocessing-api' assembly
docker: error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.31/containers/8235f0dbbe76ee195c02c98a5b1f94c87ec24f5fd4ff1c5c2e0953390534c992/start: EOF.
ERRO[0053] error waiting for container: context canceled
kj: callProcess: /Users/ttuhinanshu/dev/mmwg/scripts/cibuild (exit 125): failed

I think I'd prefer it to be native rather than Dockerized too.

kellyi commented 6 years ago

re: name:

Previously we'd been running ./sbt assemby without any arguments to specify which sub-project to build -- which meant that it built everything collected in the root project here: https://github.com/WikiWatershed/mmw-geoprocessing/blob/develop/project/build.scala#L86

In practice, the was only one project aggregated to root before: the summary project. Now that there are two projects in the build, we're running ./sbt "project mmw-geoprocessing-api" assembly to build only this project https://github.com/WikiWatershed/mmw-geoprocessing/blob/develop/project/build.scala#L88

Here's the cibuild line from this branch where that happens:

https://github.com/WikiWatershed/mmw-geoprocessing/pull/58/files#diff-0aef70f88df5d76f6b67e1befda524c5R8

I'll test out building the whole thing with assembly to check that that creates a jar with a server on 8090. If that works, we can just make a jar from the entire build, including the summary job, and skip renaming the api part.

kellyi commented 6 years ago

Oh right: so just running ./sbt assembly will build two jar files: one for "api" and one for "summary". We'd had it set up like this previously, but we were grabbing the "summary" jar for the release:

https://travis-ci.org/WikiWatershed/mmw-geoprocessing/builds/252561785#L1867

kellyi commented 6 years ago

Looks like we can do

language: scala
jdk:
  - openjdk8

I pushed a commit which removes the Dockerized build; here's the Travis output:

https://travis-ci.org/WikiWatershed/mmw-geoprocessing/builds/260619399

rajadain commented 6 years ago

Just conversed with @kellyi IRL, we came to these conclusions:

Don't rename project to mmw-geoprocessing-api. It is an unidiomatic name for Java or Scala projects, and if we want to we can rename the file during the deployment step.
Don't build just the project in the scripts. Until summary is deprecated, we might as well build both projects.

The reason for the previous setup being named mmw-geoprocessing-assembly and the new one api are because now there are two sub-projects, so each sub-project gets its own JAR file.

hectcastro commented 6 years ago

I agree with keeping it to api.

kellyi commented 6 years ago

Oof -- looks like there's an issue building both sub-projects right now because the build can't find a compatible version of Spark Job Server:

https://travis-ci.org/WikiWatershed/mmw-geoprocessing/builds/260640647#L1748

I'm inclined to change cibuild back to ./sbt "project api" assembly for now to avoid the error, since we'll presumably be removing SJS. There's not a specific pointed card for that but I think we should do it as part of #49

kellyi commented 6 years ago

Thanks for all your help with this! Going to merge this then make the 3.0.0-alpha release

WikiWatershed / mmw-geoprocessing