SlideRuleEarth / sliderule-python

SlideRule Earth Example Notebooks: On-demand, cloud-based processing of satellite mission data (NASA ICESat-2, GEDI, ArcticDEM/REMA, HLS)
https://slideruleearth.io/rtd/
BSD 3-Clause "New" or "Revised" License
41 stars 21 forks source link

SlideRule identifying 0 resources to process #107

Closed zachghiaccio closed 2 years ago

zachghiaccio commented 2 years ago

Hello,

I have been running into an issue when requesting IS-2 data through SlideRule. When using the querying functions, I receive a warning message with the following:

"INFO:sliderule.icesat2:Identified 0 resources to process"

It then returns an empty DataFrame. This occurs if I run my own parameters or use the Github examples. The issue extends to when I use the "icesat2.cmr()" command, which also returns an empty array or DataFrame. Any thoughts on what might be causing the problem?

Thank you!

Code:

from sliderule import icesat2
import pandas as pd
import matplotlib.pyplot as plt
import geopandas as gpd

icesat2.init("icesat2sliderule.org", verbose=True)

# Polygon centered over Caribou Creek, AK
region = [ {"lon":-147.66633, "lat": 65.114884},
           {"lon":-147.379038, "lat": 65.114884},
           {"lon":-147.379038, "lat": 65.252394},
           {"lon":-147.66633, "lat": 65.252394},
           {"lon":-147.66633, "lat": 65.114884} ]

# Request parameters
parms = {
    "poly": region,
    "srt": icesat2.SRT_LAND,
    "cnf": icesat2.CNF_SURFACE_LOW,
    "ats": 5.0,
    "len": 20.0,
    "res": 10.0,
    "maxi": 5
}

# Put queried data into a DataFrame. Currently returns an empty array regardless of prior inputs.
df = icesat2.atl06p(parms, "nsidc-s3")

# CMR test. This also returns an empty array.
granules = icesat2.cmr(polygon=region)
dshean commented 2 years ago

Thanks for reporting @zachghiaccio!

That region looks relatively small, and there are known issues with CMR failing to returning granules for small bbox: https://github.com/ICESat2-SlideRule/sliderule/issues/200. There is a project-level fix coming.

Does your query work with other services that use CMR?

zachghiaccio commented 2 years ago

Hi @dshean, I have used a polygon of similar size with icepyx, where it seems to work fine. Just in case, I increased the size of my box to the following:

region = [ {"lon":-149.489, "lat": 64.393},
           {"lon":-146.665, "lat": 64.393},
           {"lon":-146.665, "lat": 65.354},
           {"lon":-149.489, "lat": 65.354},
           {"lon":-149.489, "lat": 64.393} ]

...and it unfortunately returns the same warning message as above (0 resources to process).

jpswinski commented 2 years ago

@zachghiaccio Can you try your original request again? I experienced a similar issue this morning where a known region that we use for one of our system tests was returning 0 resources. A few minutes later it return maybe 1/5th of the resources, and then just recently, it went back to returning all of them.

From my own analysis of the situation, it looked like a temporary problem with CMR, which SlideRule relies on. I will be contacting them to see if they noticed the same thing on their side and if they have any explanation.

(Note, your second request returns more than 300 resources which is a soft limit in the Python client)

zachghiaccio commented 2 years ago

@jpswinski I attempted my request again (the same one as above), and I continue to receive the same error (no resources, no CMR returns). My version of SlideRule also appears to be up-to-date with the GitHub version.

I should also note that this was an issue that started happening at least a week ago. I used the above code back in early-June without any problems, but running the same code more recently raised the "0 resource "warning.

jpswinski commented 2 years ago

@zachghiaccio Thanks for trying the request again. On my side, I am able to see 108 resources returned for your original request. Would you be comfortable sending me a larger code snippet of what you are doing? I really want to try to recreate what you are seeing so I can debug it.

zachghiaccio commented 2 years ago

@jpswinski I dug into the SlideRule code, and I noticed that there were some inconsistencies in my version and the code that's provided on the GitHub. For some reason, conda was not updating SlideRule properly, and so a few of the functions were outdated.

I am working to uninstall/reinstall SlideRule right now (hopefully conda-forge will cooperate), so with luck that will solve the problem.

Thanks for your help!

zachghiaccio commented 2 years ago

Just giving an update after Friday: SlideRule appears to be up an running on my personal machine again after uninstalling/reinstalling Anaconda and SlideRule, in that order.