search

Unidata / MetPy

MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.
https://unidata.github.io/MetPy/
BSD 3-Clause "New" or "Revised" License
1.23k stars 411 forks source link

Derive 2d CAPE from a 3d field of pressure, temperature and humidity #480

Open adybbroe opened 7 years ago

adybbroe commented 7 years ago

Hi,

I am working on a small project where we want to derive CAPE from IASI (hyperspectral sounder satellite instrument) data. I have looked at SHARPpy, SkewT and this. Currently I can input one IASI profile and get out a CAPE value, using the SkewT package. But SkewT is not designed for doing it on many profiles in one go, so doing it on the entire satellite scan becomes inefficient (very slow). To me it looks like MetPy is also designed with single radiosonde type of data in mind. But perhaps you have some ideas/plans to extend it?

I am currently slowly adapting the functions taken from SkewT to digest arrays (2d or 1d) of profiles instead of just a single profile. Don't know yet, however, if I will succeed.

jrleeman commented 7 years ago

Hi Adam - As you point out, we currently don't support CAPE/CIN on a grid. It's on the roadmap, but we didn't know how much demand there would be given that most models output CAPE directly. We hadn't considered satellite data. How many profiles are you looking to calculate over? I can tell you from experience that the calculation is slow for large numbers of profiles, hopefully optimized in the future.

dopplershift commented 7 years ago

To echo what @jrleeman said, we're always interested in making what we have useful to more people, it's just that sometimes the technical challenges make it harder to expand. Performance is something that's on our roadmap, we just hadn't run into any users that found the current performance prohibitive.

Some of the numpy/scipy functions we use might make gridded data tricky, but it'd be good to have your feedback (or even a PR if you manage to figure it out).

grwise1 commented 4 years ago

So just to clarify this issue... I've been trying to use the metpy.calc functions to calculate severe parameters on a 2d grid, but to no luck. Bulk_shear, srh, stp, etc. Not possible? Seems like this feature would be highly desirable.

jthielen commented 4 years ago

So just to clarify this issue... I've been trying to use the metpy.calc functions to calculate severe parameters on a 2d grid, but to no luck. Bulk_shear, srh, stp, etc. Not possible? Seems like this feature would be highly desirable.

Correct, the current functionality for these severe parameters/profile calculations in MetPy is to operate solely on 1D profiles of input data. However, while it is not performant at the current point in time, you can still loop over the points in your grid in order to calculate using the 1D vertical profiles at each point. Coarsening your data to a lower resolution (if feasible for your application) would speed up the compute time. Xarray has some nice utilities for this that may be helpful.

Being able to compute these convective parameters efficiently on a grid is indeed a desired feature on MetPy's roadmap. Calculations like SRH and bulk shear may be more feasible in the near term since they do not require the iterative moist lapse rate and LCL calculations that things like CAPE do. These latter functions will likely require performance optimized low-level routines in Numba or Cython, which likely puts them further out on the roadmap.

In the meantime, I encourage you to also check out the xcape package to see if it fits with your needs.

dopplershift commented 4 years ago

Just to add to what @jthielen had to say: We really do want that capability. Unfortunately, manually looping over all the 1D profiles in a 3D grid can be slow, so we don't do that. And writing integrals (for e.g. CAPE) with varying starting and ending indices using NumPy is hard, if not impossible, so we haven't gotten that yet. But we're always happy to have the feedback that this is an important issue to solve at some point.

grwise1 commented 3 years ago

Thank you for the info, guys. I've found some better grib feeds with more extensive CAPE and helicity calculations, so I have some ways to move forward at this point. This package is really quite awesome either way.