search

NOAA-ORR-ERD / PyGnome

The General NOAA Operational Modeling Environment
https://gnome.orr.noaa.gov/doc/pygnome/index.html
Other
54 stars 44 forks source link

Enable PyGnome as a WPS process #11

Open metanomy opened 8 years ago

metanomy commented 8 years ago

For Geoserver WPS: http://docs.geoserver.org/2.7.0/user/community/scripting/py/index.html

For PyWPS-4: https://github.com/jachym/pywps-4

ChrisBarker-NOAA commented 8 years ago

Hmm -- only a quick look at what WPS is, but PyGNOME is a very complicated system that doesn't really fit into the GIS data processing model. So I'm not at all sure how it could be shoehorned into WPS. -- or if it would be useful.

Maybe you've got something in mind?

metanomy commented 8 years ago

At a very basic (and simplistic) level, PyGnome accepts some user-defined inputs (location, amount/quantity of oil, wind), and returns a trajectory model. Embedding PyGnome (or perhaps it's really the WebGnome API) in WPS simply gives a WPS client the ability to provide a basic popup for the user to enter the inputs, and after processing, provides a web-based view of the trajectory.

ChrisBarker-NOAA commented 8 years ago

Hmm -- I suppose it might be useful for pre-set-up locations. Though there are still quite alot of inputs -- kind of hard to put into one dialog...

But we're open to ideas :-)

metanomy commented 8 years ago

So let's pick a simple use case to think through/code against. Let's say the location is Tampa, FL. Once that has been selected, what's the minimum number of inputs needed to run an analysis - just location of spill, quantity/oil type, and wind direction, right?

The varsity play here is to use WPS to chain multiple Gnome runs, with different current models as inputs, so decision makers can compare the outputs. For instance, here is the output of Gnome with the pre-set data for Tampa, and a second output using the West Florida Shelf FVCOM model.

regnomekmlfortampabay.zip

ChrisBarker-NOAA commented 8 years ago

Hmm -- well, the trick here is that if you're going to compare anything, you'd better have the set-up just right. It seems this would make the most sense if you could grab a full model set-up, then run with one or two things changed. To facilitate that, we'd want a way to save and label a model "save file" on the server -- which we don't currently do -- though we intent to.

We could also get close with our current location files, once we do a bit more to automatically hook it into live data sources: i.e. have a location hooked up to the national weather service forecasts, or particular data buoy, or ???

As it stands now, you could probably do this for a location, if you only want to use a steady uniform wind, for instance, then the number of inputs could be pretty small.

This is feeling to me like something that you might want to build on top of WebGNOME, rather than into WebGNOME. Now that I think about it, you could build it on top of PyGNOME pretty easily right now.

IF you want to run it on the same server as your WMS, then it would be pretty simple Python script:

load up a save file set the couple parametrs you want: spill location, quantitiy, type

and run it.

the hard part would be building up that save file.

metanomy commented 8 years ago
  1. OK, so should we create an enhancement issue for "model save"? I'm happy to put some bounty money on that.
  2. Live data sources: I think this is the end-goal - pulling in real-time/forecast data for waves, wind, current, etc.
  3. Do you have an example of the minimum data required for such a save file?
ChrisBarker-NOAA commented 8 years ago

I'm not sure the model save issue is one anyone other than us could reasonably do. And we will. The trick is hositng it. As a gov't agency, we don't want to get in teh business of hosting other people's model runs/set-ups, etc -- to messy.

So is there anyone else that would want to host WebGNOME with their model set-ups?

as for minimum data -- that's where it gets tricky -- what you need entirely depends on what you are trying to model. In the general case:

A shoreline map Currrents Winds An estimate of diffusion

But whether any given data you feed the model are appropriate for the time and place and questions being asked requires a lot of judgement -- one reason we are wary of hosting other people's configurations.

But anyway, we have the currently available locations, and we will be moving toward tying them to nowcast/forecast models as we move forward.

-CHB

metanomy commented 8 years ago
  1. Hosting: Metanomy will to host a version of WebGnome as part of our EHZRAP project. While hosted version is focused on the state of Florida, we would welcome hosting models for others.
  2. Shoreline map: do you have shoreline data set for the US that is acceptable (and public)?
  3. Currents: We're assuming this data would come from forecast models (available via publicly accessible THREDDS servers).
  4. Winds: Is there a particular source of wind data you rely on (again, thinking US-specific)?
  5. Diffusion estimate: is this a quality of the type of oil being simulated?
  6. Judgement: I understand the concern. My thought is by Metanomy hosting a version (with the standard caveats) we allow people to explore, but without the risk that anyone could misconstrue it as "official" US data.
  7. Locations/forecast models: Please let us know how we can collaborate or help on this process...
ChrisBarker-NOAA commented 8 years ago

Hosting: Metanomy will to host a version of WebGnome as part of our EHZRAP project. While hosted version is focused on the state of Florida, we would welcome hosting models for others.

Awesome!

Shoreline map: do you have shoreline data set for the US that is acceptable (and public)?

Yup -- it's all on GOODS:

http://gnome.orr.noaa.gov/goods/tools/NOS/coast_subset

We are working on a web API for that, too.

Currents: We're assuming this data would come from forecast models (available via publicly accessible THREDDS servers).

Might want to run them through GOODS, too -- despite the theoretical standards, everyone seems to not-quite conform -- so GOODS converts/translates, etc for you. Also, PY_GNOME can not neccessarioy read even CF-compliant data. We're working on a better Web API for that, too.

Winds: Is there a particular source of wind data you rely on (again, thinking US-specific)?

We either use the forecast models form NOAA, or point forecast winds. See the Winds section here:

http://gnome.orr.noaa.gov/goods

Which to use is a judgment call, depending on what you are trying to model...

Diffusion estimate: is this a quality of the type of oil being simulated?

no -- a quality of the currents on the region, and the model being used. In essentially, diffusion is a result of all the detailed circulation that is not captured in the model -- so it is a function of both the circulation in the region, and how the model is simulating it..

Judgement: I understand the concern. My thought is by Metanomy hosting a version (with the standard caveats) we allow people to explore, but without the risk that anyone could misconstrue it as "official" US data.

yup -- that's better -- exactly why we have to be very careful of what we host.

Locations/forecast models: Please let us know how we can collaborate or help on this process...

I'd say getting a webGNOME server up and running and then trying to use it the way you want -- we'll then find holes in the API that we can talk about how to fill.