SlideRuleEarth / sliderule

Server and client framework for on-demand science data processing in the cloud
https://slideruleearth.io
Other
25 stars 11 forks source link

[Asset Request] ATL13 Inland Water Surface Elevations #385

Open jameshgrn opened 3 months ago

jameshgrn commented 3 months ago

Hi team,

SlideRule is so great, thanks a million for the dedicated maintenance and improvements. I'm curious if the team is looking to expand the assets available via sliderule? Specifically, I was looking at ATL13 water surface heights. If I can help put this together I'm more than happy to assist, it seems like 99% of the infra exists already but i may be missing something (or several...)

be well

Jake

jpswinski commented 3 months ago

@jameshgrn Thanks for reaching out!

What kind of functionality are you looking for, for ATL13? We could probably provide a subsetter fairly quickly - something that used segment_lat and segment_lon for the geometry. We would need you to provide us a list of variables from each track that you'd want to see and you tihnk represent a good baseline for others as well. (SlideRule hardcodes an initial set of variables to be read from the granule, which are optimized; and then the other variables can be read via the ancillary fields parameter at a slight performance cost).

jameshgrn commented 3 months ago

@jpswinski thanks a bunch for the quick response. A simple subsetter is all that I'm looking for: I have a lot of sliderule based architecture that I'd like to re-use for this ATL13 project. I've been putting together a proof of concept this weekend by hacking together fsspec and h5py and I think I've figured out which data would be most useful (at least to start!)

hdf[f'{beam_name}/ht_water_surf'][:] hdf[f'{beam_name}/ht_ortho'][:] hdf[f'{beam_name}/ht_water_surf'][:] hdf[f'{beam_name}/segment_lat'][:] hdf[f'{beam_name}/segment_lon'][:] hdf[f'{beam_name}/delta_time'][:] # Time in seconds since ATLAS SDP epoch hdf[f'{beam_name}/segment_azimuth'][:] hdf[f'{beam_name}/segment_quality'][:] (* note: this needs to be ingested as a separate dataframe and filtered) hdf[f'{beam_name}/segment_slope_trk_bdy'][:] # Slope along-track hdf[f'{beam_name}/water_depth'][:]

these seem like reasonable places to start. Again let me know if I can be of any more assisstance, thanks a bunch!

jpswinski commented 3 months ago

@jameshgrn This list looks good! Thanks for putting it together. I will try to get something together next week. We will start with an undocumented "feature preview". Then once you've used it for a little while and like how it is working, we can make it official and put it in the documentation. Until then, we can capture any changes that need to be made here in this issue.

jpswinski commented 2 months ago

@jameshgrn - an initial ATL13 subsetter is checked in on the atl13 branch and is working. All the functionality should be in there except for the segment_quality filtering.

Can you describe how you would like the segment_quality filtering to work. I see it is a 4 column table with: Column 1 - nominal Column 2 - after pulse Column 3 - impulse Column 4 - TEP

How do you want to filter the results based on these values?