search

WikiWatershed / mmw-geoprocessing

A Spark Job Server job for Model My Watershed geoprocessing.
Apache License 2.0
6 stars 6 forks source link

Mock New MultiOperation Endpoint #84

Closed rajadain closed 6 years ago

rajadain commented 6 years ago

Overview

The new multi/ endpoint will take a parent shape and a list of child shapes, as well as a list of operations, and perform all of these operations for all these shapes and return a single blob with all values.

The actual implementation will follow, but this is to solidify an interface, unblocking development in the main repo.

Connects #82

Demo

```bash $ http :8090/multi < examples/MultiOperationRequest.json ``` ```http HTTP/1.1 200 OK Content-Length: 1145154 Content-Type: application/json Date: Mon, 12 Mar 2018 19:32:19 GMT Server: akka-http/10.0.9 { "huc12_54273": { "avg_awc": { "List(0)": 8.8224446790259 }, "gwn": { "List(-256)": 80230.0, "List(0)": 1277530.0, "List(1)": 786036.0, "List(10)": 24913.0, "List(11)": 24840.0, "List(12)": 43734.0, "List(13)": 63094.0, "List(14)": 5530.0, "List(15)": 5454.0, "List(16)": 3503.0, "List(17)": 5578.0, "List(18)": 1581.0, "List(2)": 780608.0, "List(20)": 37.0, "List(22)": 1089.0, "List(23)": 1586.0, "List(27)": 14.0, "List(28)": 119.0, "List(3)": 695603.0, "List(31)": 588.0, "List(34)": 423.0, "List(35)": 2175.0, "List(4)": 504678.0, "List(46)": 224.0, "List(5)": 372267.0, "List(6)": 376547.0, "List(7)": 281170.0, "List(8)": 111029.0, "List(9)": 52278.0 }, "nlcd_kfactor": { "List(0)": 0.20602414696397067, "List(11)": 0.05508165069169296, "List(21)": 0.21868470419874417, "List(22)": 0.16695226874641336, "List(23)": 0.112148292194999, "List(24)": 0.06672464966401591, "List(31)": 0.07092705045562692, "List(41)": 0.18912035761808532, "List(42)": 0.206511688111982, "List(43)": 0.19906280961858158, "List(52)": 0.2698116178955606, "List(71)": 0.22942145138570946, "List(81)": 0.23793733590310687, "List(82)": 0.2559927368472959, "List(90)": 0.31187355224088653, "List(95)": 0.24373768152729858 }, "nlcd_slope": { "List(0, 0)": 35.0, "List(0, 1)": 123.0, "List(0, 10)": 66.0, "List(0, 12)": 47.0, "List(0, 14)": 35.0, "List(0, 15)": 20.0, "List(0, 17)": 25.0, "List(0, 19)": 22.0, "List(0, 21)": 15.0, "List(0, 23)": 10.0, "List(0, 24)": 4.0, "List(0, 26)": 2.0, "List(0, 28)": 7.0, "List(0, 3)": 144.0, "List(0, 30)": 2.0, "List(0, 32)": 3.0, "List(0, 34)": 2.0, "List(0, 36)": 3.0, "List(0, 38)": 3.0, "List(0, 40)": 3.0, "List(0, 42)": 2.0, "List(0, 44)": 2.0, "List(0, 46)": 4.0, "List(0, 5)": 126.0, "List(0, 6)": 131.0, "List(0, 8)": 92.0, "List(11, 0)": 27052.0, "List(11, 1)": 8471.0, "List(11, 10)": 1614.0, "List(11, 100)": 22.0, "List(11, 103)": 9.0, "List(11, 107)": 12.0, "List(11, 111)": 4.0, "List(11, 115)": 6.0, "List(11, 119)": 3.0, "List(11, 12)": 1192.0, "List(11, 123)": 2.0, "List(11, 127)": 5.0, "List(11, 137)": 4.0, "List(11, 14)": 910.0, "List(11, 142)": 3.0, "List(11, 148)": 2.0, "List(11, 15)": 755.0, "List(11, 153)": 3.0, "List(11, 160)": 3.0, "List(11, 166)": 4.0, "List(11, 17)": 609.0, "List(11, 19)": 523.0, "List(11, 21)": 454.0, "List(11, 23)": 393.0, "List(11, 24)": 334.0, "List(11, 26)": 285.0, "List(11, 28)": 261.0, "List(11, 3)": 5326.0, "List(11, 30)": 243.0, "List(11, 32)": 200.0, "List(11, 34)": 178.0, "List(11, 36)": 142.0, "List(11, 38)": 145.0, "List(11, 40)": 121.0, "List(11, 42)": 116.0, "List(11, 44)": 101.0, "List(11, 46)": 109.0, "List(11, 48)": 96.0, "List(11, 5)": 3873.0, "List(11, 50)": 81.0, "List(11, 53)": 83.0, "List(11, 55)": 87.0, "List(11, 57)": 68.0, "List(11, 6)": 2627.0, "List(11, 60)": 68.0, "List(11, 62)": 55.0, "List(11, 64)": 62.0, "List(11, 67)": 46.0, "List(11, 70)": 54.0, "List(11, 72)": 43.0, "List(11, 75)": 47.0, "List(11, 78)": 22.0, "List(11, 8)": 2122.0, "List(11, 80)": 27.0, "List(11, 83)": 27.0, "List(11, 86)": 18.0, "List(11, 90)": 23.0, "List(11, 93)": 21.0, "List(11, 96)": 14.0, "List(21, 0)": 59333.0, "List(21, 1)": 152659.0, "List(21, 10)": 43383.0, "List(21, 100)": 1.0, "List(21, 103)": 1.0, "List(21, 107)": 1.0, "List(21, 111)": 1.0, "List(21, 119)": 1.0, "List(21, 12)": 30867.0, "List(21, 123)": 3.0, "List(21, 132)": 2.0, "List(21, 14)": 22126.0, "List(21, 148)": 1.0, "List(21, 15)": 16554.0, "List(21, 17)": 12237.0, "List(21, 19)": 9567.0, "List(21, 21)": 6774.0, "List(21, 23)": 5298.0, "List(21, 24)": 4101.0, "List(21, 26)": 3236.0, "List(21, 28)": 2536.0, "List(21, 3)": 145563.0, "List(21, 30)": 2037.0, "List(21, 32)": 1655.0, "List(21, 34)": 1280.0, "List(21, 36)": 998.0, "List(21, 38)": 819.0, "List(21, 40)": 641.0, "List(21, 42)": 531.0, "List(21, 44)": 394.0, "List(21, 46)": 309.0, "List(21, 48)": 251.0, "List(21, 5)": 121976.0, "List(21, 50)": 202.0, "List(21, 53)": 194.0, "List(21, 55)": 136.0, "List(21, 57)": 105.0, "List(21, 6)": 85060.0, "List(21, 60)": 92.0, "List(21, 62)": 75.0, "List(21, 64)": 45.0, "List(21, 67)": 40.0, "List(21, 70)": 31.0, "List(21, 72)": 21.0, "List(21, 75)": 12.0, "List(21, 78)": 14.0, "List(21, 8)": 63736.0, "List(21, 80)": 6.0, "List(21, 83)": 11.0, "List(21, 86)": 5.0, "List(21, 90)": 5.0, "List(21, 96)": 1.0, "List(22, 0)": 42434.0, "List(22, 1)": 101217.0, "List(22, 10)": 14765.0, "List(22, 100)": 1.0, "List(22, 103)": 1.0, "List(22, 107)": 3.0, "List(22, 111)": 4.0, "List(22, 115)": 1.0, "List(22, 12)": 9377.0, "List(22, 127)": 1.0, "List(22, 14)": 6092.0, "List(22, 142)": 1.0, "List(22, 15)": 4205.0, "List(22, 17)": 2801.0, "List(22, 19)": 2015.0, "List(22, 21)": 1351.0, "List(22, 23)": 1000.0, "List(22, 24)": 677.0, "List(22, 26)": 514.0, "List(22, 28)": 388.0, "List(22, 3)": 83599.0, "List(22, 30)": 299.0, "List(22, 32)": 253.0, "List(22, 34)": 162.0, "List(22, 36)": 170.0, "List(22, 38)": 112.0, "List(22, 40)": 66.0, "List(22, 42)": 78.0, "List(22, 44)": 73.0, "List(22, 46)": 50.0, "List(22, 48)": 44.0, "List(22, 5)": 60725.0, "List(22, 50)": 33.0, "List(22, 53)": 38.0, "List(22, 55)": 22.0, "List(22, 57)": 24.0, "List(22, 6)": 36559.0, "List(22, 60)": 21.0, "List(22, 62)": 13.0, "List(22, 64)": 11.0, "List(22, 67)": 10.0, "List(22, 70)": 4.0, "List(22, 72)": 9.0, "List(22, 75)": 4.0, "List(22, 78)": 5.0, "List(22, 8)": 24185.0, "List(22, 80)": 6.0, "List(22, 83)": 4.0, "List(22, 86)": 3.0, "List(22, 90)": 2.0, "List(22, 93)": 3.0, "List(23, 0)": 36562.0, "List(23, 1)": 61184.0, "List(23, 10)": 5955.0, "List(23, 100)": 2.0, "List(23, 103)": 3.0, "List(23, 111)": 1.0, "List(23, 115)": 1.0, "List(23, 12)": 3647.0, "List(23, 123)": 1.0, "List(23, 14)": 2214.0, "List(23, 15)": 1427.0, "List(23, 17)": 1064.0, "List(23, 19)": 734.0, "List(23, 21)": 571.0, "List(23, 23)": 387.0, "List(23, 24)": 330.0, "List(23, 26)": 203.0, "List(23, 28)": 148.0, "List(23, 3)": 41867.0, "List(23, 30)": 132.0, "List(23, 32)": 100.0, "List(23, 34)": 91.0, "List(23, 36)": 83.0, "List(23, 38)": 61.0, "List(23, 40)": 48.0, "List(23, 42)": 36.0, "List(23, 44)": 40.0, "List(23, 46)": 31.0, "List(23, 48)": 23.0, "List(23, 5)": 27713.0, "List(23, 50)": 31.0, "List(23, 53)": 20.0, "List(23, 55)": 19.0, "List(23, 57)": 11.0, "List(23, 6)": 15734.0, "List(23, 60)": 23.0, "List(23, 62)": 15.0, "List(23, 64)": 24.0, "List(23, 67)": 13.0, "List(23, 70)": 13.0, "List(23, 72)": 17.0, "List(23, 75)": 14.0, "List(23, 78)": 9.0, "List(23, 8)": 9897.0, "List(23, 80)": 13.0, "List(23, 83)": 7.0, "List(23, 86)": 4.0, "List(23, 90)": 2.0, "List(23, 93)": 1.0, "List(23, 96)": 2.0, "List(24, 0)": 34720.0, "List(24, 1)": 34697.0, "List(24, 10)": 1260.0, "List(24, 107)": 1.0, "List(24, 12)": 727.0, "List(24, 14)": 405.0, "List(24, 15)": 275.0, "List(24, 17)": 184.0, "List(24, 19)": 139.0, "List(24, 21)": 78.0, "List(24, 23)": 58.0, "List(24, 24)": 50.0, "List(24, 26)": 38.0, "List(24, 28)": 29.0, "List(24, 3)": 15951.0, "List(24, 30)": 36.0, "List(24, 32)": 22.0, "List(24, 34)": 27.0, "List(24, 36)": 20.0, "List(24, 38)": 22.0, "List(24, 40)": 20.0, "List(24, 42)": 23.0, "List(24, 44)": 16.0, "List(24, 46)": 22.0, "List(24, 48)": 20.0, "List(24, 5)": 8115.0, "List(24, 50)": 17.0, "List(24, 53)": 17.0, "List(24, 55)": 13.0, "List(24, 57)": 8.0, "List(24, 6)": 4111.0, "List(24, 60)": 11.0, "List(24, 62)": 20.0, "List(24, 64)": 16.0, "List(24, 67)": 15.0, "List(24, 70)": 9.0, "List(24, 72)": 11.0, "List(24, 75)": 11.0, "List(24, 78)": 4.0, "List(24, 8)": 2370.0, "List(24, 80)": 8.0, "List(24, 83)": 1.0, "List(24, 86)": 4.0, "List(24, 90)": 4.0, "List(24, 96)": 2.0, "List(31, 0)": 1894.0, "List(31, 1)": 3413.0, "List(31, 10)": 1681.0, "List(31, 100)": 17.0, "List(31, 103)": 18.0, "List(31, 107)": 10.0, "List(31, 111)": 7.0, "List(31, 115)": 6.0, "List(31, 119)": 11.0, "List(31, 12)": 1517.0, "List(31, 123)": 11.0, "List(31, 127)": 8.0, "List(31, 132)": 5.0, "List(31, 137)": 4.0, "List(31, 14)": 1342.0, "List(31, 142)": 4.0, "List(31, 148)": 4.0, "List(31, 15)": 1160.0, "List(31, 153)": 3.0, "List(31, 160)": 2.0, "List(31, 17)": 1121.0, "List(31, 173)": 1.0, "List(31, 19)": 1087.0, "List(31, 21)": 878.0, "List(31, 23)": 778.0, "List(31, 24)": 677.0, "List(31, 26)": 603.0, "List(31, 28)": 521.0, "List(31, 3)": 3000.0, "List(31, 30)": 454.0, "List(31, 32)": 411.0, "List(31, 34)": 372.0, "List(31, 36)": 328.0, "List(31, 38)": 329.0, "List(31, 40)": 290.0, "List(31, 42)": 292.0, "List(31, 44)": 256.0, "List(31, 46)": 235.0, "List(31, 48)": 264.0, "List(31, 5)": 2659.0, "List(31, 50)": 181.0, "List(31, 53)": 177.0, "List(31, 55)": 156.0, "List(31, 57)": 152.0, "List(31, 6)": 2089.0, "List(31, 60)": 131.0, "List(31, 62)": 117.0, "List(31, 64)": 109.0, "List(31, 67)": 92.0, "List(31, 70)": 71.0, "List(31, 72)": 63.0, "List(31, 75)": 58.0, "List(31, 78)": 44.0, "List(31, 8)": 1943.0, "List(31, 80)": 46.0, "List(31, 83)": 40.0, "List(31, 86)": 46.0, "List(31, 90)": 44.0, "List(31, 93)": 26.0, "List(31, 96)": 14.0, "List(41, 0)": 50473.0, "List(41, 1)": 127164.0, "List(41, 10)": 150653.0, "List(41, 100)": 7.0, "List(41, 103)": 4.0, "List(41, 107)": 3.0, "List(41, 111)": 3.0, "List(41, 115)": 1.0, "List(41, 119)": 4.0, "List(41, 12)": 136713.0, "List(41, 123)": 3.0, "List(41, 127)": 3.0, "List(41, 132)": 2.0, "List(41, 137)": 3.0, "List(41, 14)": 121800.0, "List(41, 142)": 2.0, "List(41, 15)": 109576.0, "List(41, 17)": 91838.0, "List(41, 19)": 80243.0, "List(41, 21)": 65247.0, "List(41, 23)": 55701.0, "List(41, 24)": 46707.0, "List(41, 26)": 39174.0, "List(41, 28)": 33207.0, "List(41, 3)": 154174.0, "List(41, 30)": 28684.0, "List(41, 32)": 23709.0, "List(41, 34)": 19611.0, "List(41, 36)": 16668.0, "List(41, 38)": 13970.0, "List(41, 40)": 11403.0, "List(41, 42)": 9520.0, "List(41, 44)": 7829.0, "List(41, 46)": 6422.0, "List(41, 48)": 5145.0, "List(41, 5)": 175005.0, "List(41, 50)": 4281.0, "List(41, 53)": 3563.0, "List(41, 55)": 2618.0, "List(41, 57)": 2129.0, "List(41, 6)": 165001.0, "List(41, 60)": 1608.0, "List(41, 62)": 1234.0, "List(41, 64)": 775.0, "List(41, 67)": 601.0, "List(41, 70)": 402.0, "List(41, 72)": 241.0, "List(41, 75)": 153.0, "List(41, 78)": 95.0, "List(41, 8)": 167101.0, "List(41, 80)": 46.0, "List(41, 83)": 41.0, "List(41, 86)": 16.0, "List(41, 90)": 12.0, "List(41, 93)": 10.0, "List(41, 96)": 9.0, "List(42, 0)": 2404.0, "List(42, 1)": 6589.0, "List(42, 10)": 3415.0, "List(42, 12)": 2875.0, "List(42, 14)": 2408.0, "List(42, 15)": 2128.0, "List(42, 17)": 1831.0, "List(42, 19)": 1637.0, "List(42, 21)": 1318.0, "List(42, 23)": 1139.0, "List(42, 24)": 961.0, "List(42, 26)": 806.0, "List(42, 28)": 667.0, "List(42, 3)": 7175.0, "List(42, 30)": 576.0, "List(42, 32)": 480.0, "List(42, 34)": 354.0, "List(42, 36)": 291.0, "List(42, 38)": 201.0, "List(42, 40)": 171.0, "List(42, 42)": 136.0, "List(42, 44)": 120.0, "List(42, 46)": 80.0, "List(42, 48)": 71.0, "List(42, 5)": 6732.0, "List(42, 50)": 55.0, "List(42, 53)": 43.0, "List(42, 55)": 44.0, "List(42, 57)": 31.0, "List(42, 6)": 4985.0, "List(42, 60)": 21.0, "List(42, 62)": 18.0, "List(42, 64)": 11.0, "List(42, 67)": 14.0, "List(42, 70)": 9.0, "List(42, 72)": 8.0, "List(42, 75)": 3.0, "List(42, 78)": 3.0, "List(42, 8)": 4234.0, "List(42, 93)": 2.0, "List(43, 0)": 2344.0, "List(43, 1)": 6399.0, "List(43, 10)": 4872.0, "List(43, 103)": 2.0, "List(43, 12)": 4116.0, "List(43, 14)": 3416.0, "List(43, 15)": 2996.0, "List(43, 17)": 2534.0, "List(43, 19)": 2258.0, "List(43, 21)": 1969.0, "List(43, 23)": 1661.0, "List(43, 24)": 1560.0, "List(43, 26)": 1380.0, "List(43, 28)": 1123.0, "List(43, 3)": 8028.0, "List(43, 30)": 1123.0, "List(43, 32)": 882.0, "List(43, 34)": 749.0, "List(43, 36)": 682.0, "List(43, 38)": 559.0, "List(43, 40)": 484.0, "List(43, 42)": 375.0, "List(43, 44)": 299.0, "List(43, 46)": 242.0, "List(43, 48)": 241.0, "List(43, 5)": 7877.0, "List(43, 50)": 190.0, "List(43, 53)": 191.0, "List(43, 55)": 119.0, "List(43, 57)": 99.0, "List(43, 6)": 6533.0, "List(43, 60)": 89.0, "List(43, 62)": 58.0, "List(43, 64)": 51.0, "List(43, 67)": 47.0, "List(43, 70)": 24.0, "List(43, 72)": 25.0, "List(43, 75)": 26.0, "List(43, 78)": 17.0, "List(43, 8)": 6129.0, "List(43, 80)": 10.0, "List(43, 83)": 6.0, "List(43, 86)": 3.0, "List(43, 93)": 1.0, "List(52, 0)": 9495.0, "List(52, 1)": 26629.0, "List(52, 10)": 15505.0, "List(52, 12)": 11994.0, "List(52, 14)": 8766.0, "List(52, 15)": 6592.0, "List(52, 17)": 4787.0, "List(52, 19)": 3504.0, "List(52, 21)": 2405.0, "List(52, 23)": 1704.0, "List(52, 24)": 1274.0, "List(52, 26)": 888.0, "List(52, 28)": 670.0, "List(52, 3)": 30732.0, "List(52, 30)": 516.0, "List(52, 32)": 327.0, "List(52, 34)": 236.0, "List(52, 36)": 171.0, "List(52, 38)": 168.0, "List(52, 40)": 112.0, "List(52, 42)": 91.0, "List(52, 44)": 76.0, "List(52, 46)": 65.0, "List(52, 48)": 47.0, "List(52, 5)": 30975.0, "List(52, 50)": 52.0, "List(52, 53)": 35.0, "List(52, 55)": 24.0, "List(52, 57)": 31.0, "List(52, 6)": 25172.0, "List(52, 60)": 15.0, "List(52, 62)": 7.0, "List(52, 64)": 15.0, "List(52, 67)": 9.0, "List(52, 70)": 3.0, "List(52, 72)": 4.0, "List(52, 75)": 1.0, "List(52, 78)": 2.0, "List(52, 8)": 20991.0, "List(52, 80)": 1.0, "List(52, 83)": 1.0, "List(71, 0)": 1960.0, "List(71, 1)": 4866.0, "List(71, 10)": 2042.0, "List(71, 12)": 1494.0, "List(71, 14)": 1245.0, "List(71, 15)": 1016.0, "List(71, 17)": 814.0, "List(71, 19)": 639.0, "List(71, 21)": 480.0, "List(71, 23)": 459.0, "List(71, 24)": 350.0, "List(71, 26)": 234.0, "List(71, 28)": 175.0, "List(71, 3)": 5037.0, "List(71, 30)": 125.0, "List(71, 32)": 84.0, "List(71, 34)": 58.0, "List(71, 36)": 70.0, "List(71, 38)": 34.0, "List(71, 40)": 36.0, "List(71, 42)": 33.0, "List(71, 44)": 33.0, "List(71, 46)": 12.0, "List(71, 48)": 7.0, "List(71, 5)": 4265.0, "List(71, 50)": 9.0, "List(71, 53)": 5.0, "List(71, 55)": 4.0, "List(71, 57)": 7.0, "List(71, 6)": 3447.0, "List(71, 60)": 6.0, "List(71, 62)": 4.0, "List(71, 64)": 6.0, "List(71, 70)": 3.0, "List(71, 72)": 1.0, "List(71, 78)": 1.0, "List(71, 8)": 2886.0, "List(81, 0)": 53296.0, "List(81, 1)": 139765.0, "List(81, 10)": 64754.0, "List(81, 103)": 1.0, "List(81, 119)": 1.0, "List(81, 12)": 48112.0, "List(81, 123)": 1.0, "List(81, 14)": 34735.0, "List(81, 15)": 25225.0, "List(81, 17)": 17333.0, "List(81, 19)": 11860.0, "List(81, 21)": 7576.0, "List(81, 23)": 4880.0, "List(81, 24)": 3147.0, "List(81, 26)": 1893.0, "List(81, 28)": 1214.0, "List(81, 3)": 144506.0, "List(81, 30)": 842.0, "List(81, 32)": 489.0, "List(81, 34)": 282.0, "List(81, 36)": 251.0, "List(81, 38)": 136.0, "List(81, 40)": 111.0, "List(81, 42)": 74.0, "List(81, 44)": 68.0, "List(81, 46)": 47.0, "List(81, 48)": 22.0, "List(81, 5)": 136552.0, "List(81, 50)": 22.0, "List(81, 53)": 14.0, "List(81, 55)": 5.0, "List(81, 57)": 9.0, "List(81, 6)": 104712.0, "List(81, 60)": 7.0, "List(81, 62)": 7.0, "List(81, 67)": 1.0, "List(81, 70)": 1.0, "List(81, 72)": 1.0, "List(81, 78)": 1.0, "List(81, 8)": 86028.0, "List(81, 80)": 1.0, "List(81, 86)": 3.0, "List(82, 0)": 68162.0, "List(82, 1)": 150234.0, "List(82, 10)": 31118.0, "List(82, 107)": 1.0, "List(82, 12)": 20378.0, "List(82, 132)": 1.0, "List(82, 14)": 13127.0, "List(82, 15)": 9115.0, "List(82, 153)": 1.0, "List(82, 17)": 5726.0, "List(82, 19)": 3478.0, "List(82, 21)": 2092.0, "List(82, 23)": 1324.0, "List(82, 24)": 779.0, "List(82, 26)": 441.0, "List(82, 28)": 281.0, "List(82, 3)": 127994.0, "List(82, 30)": 168.0, "List(82, 32)": 112.0, "List(82, 34)": 85.0, "List(82, 36)": 57.0, "List(82, 38)": 32.0, "List(82, 40)": 49.0, "List(82, 42)": 33.0, "List(82, 44)": 24.0, "List(82, 46)": 16.0, "List(82, 48)": 18.0, "List(82, 5)": 100586.0, "List(82, 50)": 13.0, "List(82, 53)": 10.0, "List(82, 55)": 10.0, "List(82, 57)": 12.0, "List(82, 6)": 67283.0, "List(82, 60)": 3.0, "List(82, 62)": 8.0, "List(82, 64)": 2.0, "List(82, 67)": 2.0, "List(82, 70)": 1.0, "List(82, 72)": 3.0, "List(82, 75)": 1.0, "List(82, 78)": 3.0, "List(82, 8)": 47868.0, "List(82, 80)": 2.0, "List(90, 0)": 25099.0, "List(90, 1)": 21946.0, "List(90, 10)": 1134.0, "List(90, 12)": 804.0, "List(90, 14)": 525.0, "List(90, 15)": 345.0, "List(90, 17)": 263.0, "List(90, 19)": 171.0, "List(90, 21)": 136.0, "List(90, 23)": 116.0, "List(90, 24)": 73.0, "List(90, 26)": 53.0, "List(90, 28)": 46.0, "List(90, 3)": 10030.0, "List(90, 30)": 43.0, "List(90, 32)": 35.0, "List(90, 34)": 26.0, "List(90, 36)": 26.0, "List(90, 38)": 26.0, "List(90, 40)": 22.0, "List(90, 42)": 19.0, "List(90, 44)": 14.0, "List(90, 46)": 10.0, "List(90, 48)": 6.0, "List(90, 5)": 6302.0, "List(90, 50)": 11.0, "List(90, 53)": 11.0, "List(90, 55)": 5.0, "List(90, 57)": 3.0, "List(90, 6)": 3490.0, "List(90, 60)": 8.0, "List(90, 62)": 8.0, "List(90, 64)": 6.0, "List(90, 67)": 5.0, "List(90, 70)": 9.0, "List(90, 72)": 3.0, "List(90, 75)": 6.0, "List(90, 78)": 7.0, "List(90, 8)": 1980.0, "List(90, 80)": 4.0, "List(90, 83)": 5.0, "List(90, 86)": 1.0, "List(90, 90)": 2.0, "List(90, 93)": 1.0, "List(95, 0)": 1502.0, "List(95, 1)": 1133.0, "List(95, 10)": 114.0, "List(95, 12)": 89.0, "List(95, 14)": 77.0, "List(95, 15)": 42.0, "List(95, 17)": 29.0, "List(95, 19)": 32.0, "List(95, 21)": 23.0, "List(95, 23)": 17.0, "List(95, 24)": 8.0, "List(95, 26)": 12.0, "List(95, 28)": 9.0, "List(95, 3)": 597.0, "List(95, 30)": 7.0, "List(95, 32)": 5.0, "List(95, 34)": 4.0, "List(95, 36)": 4.0, "List(95, 38)": 6.0, "List(95, 40)": 3.0, "List(95, 42)": 2.0, "List(95, 44)": 1.0, "List(95, 46)": 1.0, "List(95, 48)": 4.0, "List(95, 5)": 430.0, "List(95, 6)": 280.0, "List(95, 8)": 196.0 }, "nlcd_soils": { "List(0, -2147483648)": 174.0, "List(0, 1)": 149.0, "List(0, 2)": 437.0, "List(0, 3)": 77.0, "List(0, 4)": 37.0, "List(0, 6)": 49.0, "List(0, 7)": 5.0, "List(11, -2147483648)": 44525.0, "List(11, 1)": 4231.0, "List(11, 2)": 4954.0, "List(11, 3)": 1229.0, "List(11, 4)": 1846.0, "List(11, 6)": 1966.0, "List(11, 7)": 429.0, "List(21, -2147483648)": 218609.0, "List(21, 1)": 94657.0, "List(21, 2)": 240211.0, "List(21, 3)": 87922.0, "List(21, 4)": 89381.0, "List(21, 6)": 17228.0, "List(21, 7)": 46919.0, "List(22, -2147483648)": 213390.0, "List(22, 1)": 19717.0, "List(22, 2)": 87254.0, "List(22, 3)": 27001.0, "List(22, 4)": 30903.0, "List(22, 6)": 4754.0, "List(22, 7)": 10416.0, "List(23, -2147483648)": 153940.0, "List(23, 1)": 7003.0, "List(23, 2)": 30235.0, "List(23, 3)": 7465.0, "List(23, 4)": 7611.0, "List(23, 6)": 1429.0, "List(23, 7)": 2845.0, "List(24, -2147483648)": 86936.0, "List(24, 1)": 2570.0, "List(24, 2)": 9055.0, "List(24, 3)": 2169.0, "List(24, 4)": 1945.0, "List(24, 6)": 447.0, "List(24, 7)": 485.0, "List(31, -2147483648)": 9839.0, "List(31, 1)": 15667.0, "List(31, 2)": 1930.0, "List(31, 3)": 1396.0, "List(31, 4)": 1506.0, "List(31, 6)": 138.0, "List(31, 7)": 796.0, "List(41, -2147483648)": 61560.0, "List(41, 1)": 697001.0, "List(41, 2)": 584572.0, "List(41, 3)": 169841.0, "List(41, 4)": 219019.0, "List(41, 6)": 39058.0, "List(41, 7)": 159556.0, "List(42, -2147483648)": 1821.0, "List(42, 1)": 11380.0, "List(42, 2)": 16239.0, "List(42, 3)": 4711.0, "List(42, 4)": 13093.0, "List(42, 6)": 2965.0, "List(42, 7)": 3837.0, "List(43, -2147483648)": 4071.0, "List(43, 1)": 19955.0, "List(43, 2)": 17857.0, "List(43, 3)": 7030.0, "List(43, 4)": 14639.0, "List(43, 6)": 2305.0, "List(43, 7)": 5932.0, "List(52, -2147483648)": 8573.0, "List(52, 1)": 50503.0, "List(52, 2)": 62373.0, "List(52, 3)": 23913.0, "List(52, 4)": 28662.0, "List(52, 6)": 2030.0, "List(52, 7)": 28038.0, "List(71, -2147483648)": 1565.0, "List(71, 1)": 9131.0, "List(71, 2)": 9940.0, "List(71, 3)": 2788.0, "List(71, 4)": 5299.0, "List(71, 6)": 268.0, "List(71, 7)": 2956.0, "List(81, -2147483648)": 13943.0, "List(81, 1)": 115211.0, "List(81, 2)": 470329.0, "List(81, 3)": 82961.0, "List(81, 4)": 151904.0, "List(81, 6)": 21591.0, "List(81, 7)": 32046.0, "List(82, -2147483648)": 6652.0, "List(82, 1)": 104583.0, "List(82, 2)": 356754.0, "List(82, 3)": 63108.0, "List(82, 4)": 79426.0, "List(82, 6)": 8865.0, "List(82, 7)": 31265.0, "List(90, -2147483648)": 7753.0, "List(90, 1)": 2427.0, "List(90, 2)": 7606.0, "List(90, 3)": 8340.0, "List(90, 4)": 8094.0, "List(90, 6)": 16551.0, "List(90, 7)": 22064.0, "List(95, -2147483648)": 1114.0, "List(95, 1)": 87.0, "List(95, 2)": 554.0, "List(95, 3)": 258.0, "List(95, 4)": 930.0, "List(95, 6)": 1110.0, "List(95, 7)": 574.0 }, "nlcd_streams": { "List(0)": 13.0, "List(11)": 8095.0, "List(21)": 11553.0, "List(22)": 3252.0, "List(23)": 1326.0, "List(24)": 364.0, "List(31)": 217.0, "List(41)": 33341.0, "List(42)": 2149.0, "List(43)": 2387.0, "List(52)": 2874.0, "List(71)": 150.0, "List(81)": 10301.0, "List(82)": 4736.0, "List(90)": 14005.0, "List(95)": 677.0 }, "slope": { "List(0)": 50.38815789473684 } }, ... } ```

Notes

Classically, some of the results have been Ints, but since the output is all grouped together now we bump it all to Double for consistency.

Testing Instructions

arottersman commented 6 years ago

Taking a look now

arottersman commented 6 years ago

Doesn't matter for the mocks, but would like to point out that the subbasin jobs currently use the true watershed IDs, not our table ids to identify the HUC-12s. The Drexel API expects as much, so I think we should be consistent throughout the entirety of the pipeline and avoid further name look ups.

(As an aside I hope that we some day get rid of our own table ids and switch the cache and everything over to the watershed ids as discussed in WikiWatershed/model-my-watershed#2202 and WikiWatershed/model-my-watershed#2526)

rajadain commented 6 years ago

Doesn't matter for the mocks, but would like to point out that the subbasin jobs currently use the true watershed IDs, not our table ids to identify the HUC-12s. The Drexel API expects as much, so I think we should be consistent throughout the entirety of the pipeline and avoid further name look ups.

Good to know! I was just using the cache keys. They are simple labels, and can be changed by the caller, so will change automatically when we change the cache keys themselves.

rajadain commented 6 years ago

Any sense of what the cost of doubling the payload size to send the parent shape is vs calculating the union of the childShapes?

There was some meditation on this here: https://github.com/WikiWatershed/mmw-geoprocessing/issues/82#issuecomment-371632018. I was waiting until I had an implementation of the actual endpoint to make this decision, which I now do. Will try it both ways.

I think having only one place to send shapes instead of two will make it easier to generalize this operation to support a single shape.

rajadain commented 6 years ago

Replaced parentShape and childShapes (blue) with just one shapes (red), and the timing seems generally consistent:

image

Where in the latter (red) the AoI is composed by unioning all the component shapes. The output files are also identical.

Will remove parentShape and rename childShapes to shapes.

rajadain commented 6 years ago

It should be noted that removing the parent shape does not reduce the request size by 50%. With it, the request is 3.6MB, without it 3.2MB.

rajadain commented 6 years ago

Added pixelIsArea to the operation block, to allow using it with low resolution rasters.

arottersman commented 6 years ago

It should be noted that removing the parent shape does not reduce the request size by 50%. With it, the request is 3.6MB, without it 3.2MB.

Oh, right, forgot about all the internal vertices that make up the bulk of the payload.