search

ioos / registry

Getting data services registered in the IOOS Service Registry
http://ioos.github.io/registry/
2 stars 7 forks source link

HRRR data is in Unidata ISO WAF, but not in registry #81

Open rsignell-usgs opened 9 years ago

rsignell-usgs commented 9 years ago

I'm trying to figure out why the HRRR (high-resolution rapid refresh, very cool!) model is not in the registry at http://www.ngdc.noaa.gov/geoportal

but the WAF is in the Collection Source Table

and the HRRR is in the WAF

I see the collection table says "Needs Cleanup".

Is that something we need to do on the provider side (e.g. @kwilcox), or is that on the NGDC side(e.g. @amilan17)?

amilan17 commented 9 years ago

@rsignell-usgs I think this is an issue on the NGDC side. For some reason - the Unidata records were not getting synced into the production WAF during it's scheduled time. This has been happening since 8/15. I've manually synced to EMMA and Geoportal today, it may be a larger timing issue. I'll check again tomorrow.

rsignell-usgs commented 9 years ago

Okay,thanks. I'm hoping to use these in catalog-driven workflow demos Wednesday and Friday!

rsignell-usgs commented 9 years ago

The manual syncing worked, and today the HRRR is back! @amilan17, I hope you can find out what is wrong with the automated process, but if you can't figure it out today, can you manually sync again so that I have these for tomorrow's demo?

2015-09-01_0711

amilan17 commented 9 years ago

Hi Rich, rather than manually syncing every day until we figure out what is going on - I configured a couple things:

  1. EMMA processing will NOT remove previous harvest - it will only add new and replace current files.
  2. Geoportal is set to manual syncing instead of daily. So they will not disappear if the EMMA waf is cleared out.

Will it work for you to use yesterday's version of these records for your demos this week?

Anna ~~Metadata Adds Meaning.~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA National Centers for Environmental Information formerly NOAA’s National Geophysical Data Center http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~~

On Tue, Sep 1, 2015 at 5:15 AM, Rich Signell notifications@github.com wrote:

The manual syncing worked, and today the HRRR is back! @amilan17 https://github.com/amilan17, I hope you can find out what is wrong with the automated process, but if you can't figure it out today, can you manually sync again so that I have these for tomorrow's demo?

[image: 2015-09-01_0711] https://cloud.githubusercontent.com/assets/1872600/9602762/32963aa6-5079-11e5-8539-7f8711163ead.png

— Reply to this email directly or view it on GitHub https://github.com/ioos/registry/issues/81#issuecomment-136677270.

rsignell-usgs commented 9 years ago

@amilan17, since these are short range (36 hour) forecast models, they would be updated every day, because if I search for data that contains the current forecast on Friday, I won't get records back.

amilan17 commented 9 years ago

Ok. Good to know. I'll undo previous changes.

Anna ~~Metadata Adds Meaning.~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA National Centers for Environmental Information formerly NOAA’s National Geophysical Data Center http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~~

On Tue, Sep 1, 2015 at 11:54 AM, Rich Signell notifications@github.com wrote:

@amilan17 https://github.com/amilan17, since these are short range (36 hour) forecast models, they would be updated every day, because if I search for data that contains the current forecast on Friday, I won't get records back.

— Reply to this email directly or view it on GitHub https://github.com/ioos/registry/issues/81#issuecomment-136811719.

amilan17 commented 9 years ago

Do the names of the files change every day?

Anna ~~Metadata Adds Meaning.~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA National Centers for Environmental Information formerly NOAA’s National Geophysical Data Center http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~~

On Tue, Sep 1, 2015 at 12:36 PM, Anna Milan - NOAA Federal < anna.milan@noaa.gov> wrote:

Ok. Good to know. I'll undo previous changes.

Anna ~~Metadata Adds Meaning.~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA National Centers for Environmental Information formerly NOAA’s National Geophysical Data Center http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~~

On Tue, Sep 1, 2015 at 11:54 AM, Rich Signell notifications@github.com wrote:

@amilan17 https://github.com/amilan17, since these are short range (36 hour) forecast models, they would be updated every day, because if I search for data that contains the current forecast on Friday, I won't get records back.

— Reply to this email directly or view it on GitHub https://github.com/ioos/registry/issues/81#issuecomment-136811719.

rsignell-usgs commented 9 years ago

No, thankfully the URLs stay the same! The only thing that should change in the metadata are the start/stop times!

amilan17 commented 9 years ago

OK. I configured the EMMA processing to NOT remove records before processing, so at least the WAF won't be empty... However, I'll still manually sync new records this week - unless we figure out what is going on !

Anna ~~Metadata Adds Meaning.~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA National Centers for Environmental Information formerly NOAA’s National Geophysical Data Center http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~~

On Tue, Sep 1, 2015 at 12:58 PM, Rich Signell notifications@github.com wrote:

No, thankfully the URLs stay the same! The only thing that should change in the metadata are the start/stop times!

— Reply to this email directly or view it on GitHub https://github.com/ioos/registry/issues/81#issuecomment-136828122.

rsignell-usgs commented 9 years ago

@amilan17 , okay, thanks so much! I appreciate it!