search

icesat2py / icepyx

Python tools for obtaining and working with ICESat-2 data
https://icepyx.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
208 stars 108 forks source link

Evaluate the latest CMR python library to determine if it meets the needs of icepyx #169

Open asteiker opened 3 years ago

asteiker commented 3 years ago

The CMR team has recently published a basic python library that can search collections and granules using the CMR API behind the scenes: https://github.com/nasa/eo-metadata-tools/tree/master/CMR/python

This has relatively limited functionality as an initial release but it may be worth evaluating if this type of library could suit the needs of icepyx in the future, and/or if any CMR code that has been developed under icepyx could be contributed to the CMR library so that the icepyx code becomes more simplified with less dependencies.

This basic notebook demonstrating granule search may be helpful to examine if/how we could leverage the library: https://github.com/nasa/eo-metadata-tools/blob/master/CMR/python/demos/notebooks/granules.ipynb

JessicaS11 commented 3 years ago

Thanks for sharing this great resource, @asteiker! @alexdibella, maybe we can dive into this a little during an upcoming hack!

alexdibella commented 3 years ago

Good idea @JessicaS11!! Won't be there today, but we can hack it out next week :)

wenjieji86 commented 3 years ago

Hi @JessicaS11 @asteiker this may be the wrong place to ask this question but I don't know where else to go so I will just go ahead and ask here... The CMR API allows temporal search/subsetting, but it does so by combining date and time. This means subsetting via time only is impossible (e.g. I may be interested only in night-time data from July to December 2020)? Is there a way that I can do this without downloading all available data from July to December 2020 then subsetting locally?

Thanks.

JessicaS11 commented 3 years ago

@wenjieji86 Apologies that this comment slipped under the radar. Unfortunately, I suspect the best approach is going to be to loop through a set of dates and place an order (and download) for each one. Then the temporal subsetting you're after will be accomplished. I'm not sure there's another way you could go about this, but my understanding of CMR is limited so I could be totally off base. Some discussion on automated login (which you'd probably want to use for this) is here.

@amy I looked into the CMR library a bit more, and I certainly think we could change out our current requests and manual formatting of url-based workflow to utilize this library. I probably won't have dev time to do so in the near future, but I suspect it could be a pretty quick transition for someone more familiar with CMR and access routes/using it in general, and I'd be happy to chat with someone about the needs on our side of things.

JessicaS11 commented 3 years ago

@alexdibella In follow up to your question today, see the second part of the most recent post. So if you are able to access Sentinel data through CMR and want to make the transition, go ahead, but it's not yet a planned dev step for icepyx.