search

noaa-oar-arl / monetio

The Model and ObservatioN Evaluation Tool I/O package
https://monetio.readthedocs.io
MIT License
16 stars 19 forks source link

Add Gridded VIIRS from the NOAA AWS bucket #164

Open bbakernoaa opened 3 months ago

bbakernoaa commented 3 months ago

Added a new function to stream data from AWS from the noaa-jpss bucket into xarray for monetio. This should be useful for satellite comparisons.

It can handle both SNPP and NOAA20 satellites as well as varying resolutions (0.05, 0.1, 0.25 degree) and different time aggregations that are preprocessed by NESDIS (daily, weekly, monthly).

Data availability is from 2012 -> current.

to use:

nests_viirs_aod_gridded.open_dataset('2013-01-01', 'SNPP', '0.100', 'daily')

nests_viirs_aod_gridded.open_mfdataset(dates, 'SNPP', '0.050', 'daily')

@angehung5

bbakernoaa commented 3 months ago

just a few seconds at most

bbakernoaa commented 3 months ago

Can you run the pre-commit hooks?

How long does it take to get a small amount of data (for test)?

Not quite sure how to do that. Can you provide a little instruction?

bbakernoaa commented 3 months ago

I added back in the NRT data. Seems the AWS server didn't have a the latest data from the past 3 years. in the NRT data we don't have the 0.05 data but we do have the 0.1 and 0.25 degree data.

zmoon commented 3 months ago

Not quite sure how to do that.

  1. Install pre-commit to your env if you don't have it available already (it's available on conda-forge)
  2. pre-commit install --install-hooks
  3. pre-commit run --all-files
zmoon commented 3 months ago

just a few seconds at most

Sweet, do you want a write a test? I can give it a go if not.

bbakernoaa commented 3 months ago

Not quite sure how to do that.

  1. Install pre-commit to your env if you don't have it available already (it's available on conda-forge)
  2. pre-commit install --install-hooks
  3. pre-commit run --all-files

Thanks! Added it and got these rolling. Seems the tests are failing but in unrelated spots where the code was changed