WikiWatershed / mmw-geoprocessing

A Spark Job Server job for Model My Watershed geoprocessing.
Apache License 2.0
6 stars 6 forks source link

Add soiln, soilp, recess_coeff operations #88

Closed rajadain closed 6 years ago

rajadain commented 6 years ago

Overview

Real-world MapShed requests also use these operations as part of their modeling, so to better reflect the reality of usage we add them to the example requests and responses.

Connects #83

Demo

Update benchmark:

> kj benchmark --mapshed

Timing HUC8 MultiOperation MapShed ->

HUC8 MultiOperation MapShed, run 1 -> 8.854033 s
HUC8 MultiOperation MapShed, run 2 -> 7.97886 s
HUC8 MultiOperation MapShed, run 3 -> 7.882735 s
HUC8 MultiOperation MapShed, run 4 -> 7.910098 s
HUC8 MultiOperation MapShed, run 5 -> 7.519246 s

HUC8 MultiOperation MapShed average -> 8.0289944 s

Timing HUC8 MapShed, separate requests per operation ->

HUC8 MapShed, 7 operations, run 1 -> 10.07 s
HUC8 MapShed, 7 operations, run 2 -> 10.46 s
HUC8 MapShed, 7 operations, run 3 -> 10.18 s
HUC8 MapShed, 7 operations, run 4 -> 9.74 s
HUC8 MapShed, 7 operations, run 5 -> 10.21 s

Testing Instructions

kellyi commented 6 years ago

Having a hard time getting this setup because of some weird mismatch between the pipenv and the required requests module.

Now that we've got multiple dependencies it seems like it'd benefit from locking them down to ensure that developers can install everything with a single command -- setup probably -- and also have everything installed to run the benchmark script. Would also benefit from updating the README.

That work may be out of scope for what's here, though.

rajadain commented 6 years ago

If you have pipenv installed (you can do brew install pipenv), you can just run ./scripts/setup which will install all the right dependencies.

Updating the README is a legitimate need, but out of scope (and possibly budget).

rajadain commented 6 years ago

If running the benchmark-sync script, prefer doing so with pipenv:

$ pipenv run ./benchmark-sync
rajadain commented 6 years ago

But none of the pipenv stuff should be required for testing this particular PR. curl or http should be enough.

kellyi commented 6 years ago

Makes sense. I installed pipenv after running setup but didn't re-run setup. Did that and was able to run benchmark --mapshed

rajadain commented 6 years ago

Thank you both for taking a look!