search

deeplycloudy / xlma-python

A future, Python-based version of xlma?
MIT License
6 stars 6 forks source link

Geometry operations for point data #14

Open gewitterblitz opened 3 years ago

gewitterblitz commented 3 years ago

I recently stumbled upon the spatialpandas library while exploring visualization options using datashader/holoviews.

Here's the link to the design document.

Do you think it can be useful for visualization of LMA data?

gewitterblitz commented 3 years ago

I have uploaded a use case for NLDN data to compute peak current values for IC and CG flashes restricted to the lasso boundary created using lmatools. This implementation assumes that only the storm of interest (SOI) passed through the lasso region during the course of analysis. In other words, between t_start and t_end, there were no other storms tracing the same path (or even overlapping) as the SOI. Here is the link to the jupyter notebook

gewitterblitz commented 3 years ago

Edit: Here's the overlay plot for all lasso polygons and NLDN (IC +CG flashes) after accurate count of NLDN flashes with SOI

Red: NLDN IC flashes Blue: NLDN CG flashes Opacity varies by count number in a grid cell.

bokeh_plot

gewitterblitz commented 3 years ago

Update: Analysis code in the jupyter notebook above has been modified to allow counting of only those CG/IC flashes which occurred within lasso at their respective times. Thus, we no longer have to assume that only the SOI passed through that area.

This also results in accurate CG flash rate calculation within SOI.

deeplycloudy commented 3 years ago

@gewitterblitz this is a really nice use example. At a minimum, I think it would be great to include as an example notebook showing how to do these analyses. Feel free to open a PR for the notebook.

I'm not sure how I feel about adding a spatialpandas requirement for the core library. How is it different from geopandas? We probably should have a larger design discussion about moving the lasso functionality into xlma-python, too.

gewitterblitz commented 3 years ago

Sure, I can do that. Regarding spatialpandas, this is what I found in their documents which may (or may not) be relevant while dealing with thousands (or millions) of VHF sources in LMA data:

Spatialpandas is a new library that provides Pandas and Dask extensions for vectorized spatial and geometric operations. Dask allowed us to scale the algorithms and terraform-jupyter gave us a cheap cloud autoscaling compute environment. In this talk we will demonstrate how you can do the same.

Here is the link to their SciPy 2020 talk.

I agree with your idea about having a discussion on the design/structure of xlma-python. We (all who are interested) could meet on discord. Let us know what you think.