search

ioos / registry

Getting data services registered in the IOOS Service Registry
http://ioos.github.io/registry/
2 stars 7 forks source link

Register NANOOS WMS endpoint (from GeoServer) #19

Open robragsdale opened 10 years ago

robragsdale commented 10 years ago

From: Emilio Mayorga mayorga@apl.washington.edu Date: Sunday, May 11, 2014 at 4:08 PM To: Anna Milan - NOAA Federal anna.milan@noaa.gov Cc: David Foster dfoster@asascience.com, _NOAA IOOS CATALOG IOOS.Catalog@noaa.gov Subject: Re: Register NANOOS WMS endpoint (from GeoServer)

Hi Anna et al,

I see that our WMS endpoint is now on the NGDC GeoPortal: http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={D4B1CAF9-1368-4CFF-BA1B-CF0B94FB0F3C}

But as we all might have guessed, based on the limited attention we've paid to WMS in IOOS, the information being extracted and portrayed is rather sparse.

I'm willing to work with you and others at NGDC to enhance handling of WMS endpoints, not just to benefit this NANOOS service, but to improve overall NGDC/IOOS capabilities. Let me know how I can help.

Dave, I'd also like to see when and how this makes it into the Catalog.

Can someone (Rob or Derrick?) copy this email thread into a new ioos github registry issue as well? That'll make it easier to track the details of the discussion.

Thanks! Cheers, -Emilio

robragsdale commented 10 years ago

Hi Emilio,

On the Catalog side we have also paid not enough attention to WMS, so certainly interested in leveling up there.

dave

emiliom commented 10 years ago

(Here's my original message, for reference) I'd like to register a WMS GetCapabilities endpoint from NANOOS, served from GeoServer. According to the instructions here all that's needed is the GetCapabilities endpoint. One thing that's not clear is if IOOS/NGDC have a preference for one specific WMS version -- 1.3.0 vs 1.1.1. Is it possible to provide both and have Registry/GeoPortal/IOOS Catalog system give access to both, w/o creating too much duplication and confusion? Either way, here are the endpoints for both versions: http://data.nanoos.org/geoserver/nvs/ows?service=WMS&version=1.3.0&request=GetCapabilities http://data.nanoos.org/geoserver/nvs/ows?service=WMS&version=1.1.1&request=GetCapabilities

The GetCapabilities abstract is currently way too sparse. I'll enrich it soon. But we do have rich layer titles and abstracts.

Do you have any suggestions for Keywords?

There's one small detail that's not clear to me, though it's probably specific to GeoServer. The base URL I'm providing here is what I consider the cleanest, simplest version: http://data.nanoos.org/geoserver/nvs/ows?service=WMS

However, GeoServer provides a couple of ways to point to the same thing, and the GetCapabilities document actually lists a slightly different and less clean base URL: http://data.nanoos.org:80/geoserver/nvs/wms?service=WMS

Note the unnecessary use of a port number, and of "wms?" rather than "ows?". "ows?" is preferable that allows for a consistent base URL for a WFS GetCapabilities (which I know IOOS isn't supporting yet): http://data.nanoos.org/geoserver/nvs/ows?service=WFS

I'd be very curious to know if there are other GeoServer-based WMS services in the IOOS Registry/GeoPortal/Catalog. If there are, can someone point me to examples?

I'm the point of contact for this service.

ebridger commented 10 years ago

Not sure if this is relevant or not. NERACOOS has a WMS registered in the registry and in the catalog. The NGDC link is: http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid=%7B173F0925-8031-4288-858B-26F886309FE3%7D The WMS GetCap is: http://www.neracoos.org/thredds/wms/WW3/GulfOfMaine.nc?service=WMS&version=1.3.0&request=GetCapabilities This is a WMS done by ncWMS via THREDDS. I used ncml in the TDS catalog entry to add metadata to the GetCapabilities response.

emiliom commented 10 years ago

Thanks, Eric! THREDDS ncWMS endpoints are not so useful (and there are many of them already registered) for this purpose, b/c they're typically fairly simplistic WMS use cases. But you inspired me to scan all WMS services on the Catalog, and I found more relevant examples from PacIOOS that use Mapserver; eg: http://catalog.ioos.us/services/534615d88c0db36a5ba99ccf

emiliom commented 10 years ago

@amilan17, have you had a chance to think about how the NANOOS WMS endpoint is being harvested into NGDC GeoPortal? Like I said, there's much more information in my WMS GetCapabilities that's not captured in the GeoPortal record, for example the spatial bounding box, contact information, etc. http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={D4B1CAF9-1368-4CFF-BA1B-CF0B94FB0F3C}

PacIOOS has a couple of WMS endpoints on the IOOS Catalog that seem relevant to what NANOOS is trying to do, in that they originate in "GIS servers" rather than THREDDS-ncWMS. Based on very a quick visual scan of all WMS services accessible on the IOOS Catalog, that's the only set of non-THREDDS WMS endpoints I could find. The IOOS Catalog entry for the "PacIOOS Boundaries: EEZs and Geographic" record is here: http://catalog.ioos.us/services/534615d88c0db36a5ba99ccf That's bare bones, but still useful.

I found the record in NGDC GeoPortal: http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={3CB74631-5782-43D4-9924-F09EA11F0AEA} That's more like it! The information looks rich. So I looked at the ISO metadata record and saw that it's quite rich and even includes information about the corresponding WFS endpoint, which NGDC is not handling (no worries, I know WFS is not on the target list yet). My semi-educated guess is that PacIOOS is making available via a WAF an ISO 19115-2 metadata record for this dataset, that includes both WMS and WFS endpoints; and that NGDC is harvesting that WAF metadata rather than a WMS GetCapabilities response. @pacioos John, can you confirm this? If so, that means that NANOOS would have to something "extra" to make our GeoServer WMS (and WFS?) endpoints nicely parseable by NGDC; that would be a bit disheartening, b/c it's extra work.

@pacioos, can you confirm that you're creating WAF ISO metadata records for your WMS/WFS endpoints/datasets? If so, can you tell us briefly how you generate these? Are they hand crafted, or do they involve some step that scrapes (eg, via an XSLT) information from the WMS & WFS GetCapabilities to pre-fill sections of the ISO metadata?

pacioos commented 10 years ago

Hi Emilio, Your assumptions are correct. I hand-generated the ISO metadata records for those few non-THREDDS WMS/WFS endpoints and added them to our WAF. I start with a "good" ISO record as a template, and then replace information for the new dataset. Cheers, John

On Thu, May 15, 2014 at 9:32 AM, Emilio Mayorga notifications@github.comwrote:

@amilan17 https://github.com/amilan17, have you had a chance to think about how the NANOOS WMS endpoint is being harvested into NGDC GeoPortal? Like I said, there's much more information in my WMS GetCapabilities that's not captured in the GeoPortal record, for example the spatial bounding box, contact information, etc.

http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={D4B1CAF9-1368-4CFF-BA1B-CF0B94FB0F3C}

PacIOOS has a couple of WMS endpoints on the IOOS Catalog that seem relevant to what NANOOS is trying to do, in that they originate in "GIS servers" rather than THREDDS-ncWMS. Based on very a quick visual scan of all WMS services accessible on the IOOS Cataloghttp://catalog.ioos.us/services/filter/null/WMS, that's the only set of non-THREDDS WMS endpoints I could find. The IOOS Catalog entry for the "PacIOOS Boundaries: EEZs and Geographic" record is here: http://catalog.ioos.us/services/534615d88c0db36a5ba99ccf That's bare bones, but still useful.

I found the record in NGDC GeoPortal:

http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={3CB74631-5782-43D4-9924-F09EA11F0AEA} That's more like it! The information looks rich. So I looked at the ISO metadata record and saw that it's quite rich and even includes information about the corresponding WFS endpoint, which NGDC is not handling (no worries, I know WFS is not on the target list yet). My semi-educated guess is that PacIOOS is making available via a WAF an ISO 19115-2 metadata record for this dataset, that includes both WMS and WFS endpoints; and that NGDC is harvesting that WAF metadata rather than a WMS GetCapabilities response. @pacioos https://github.com/pacioos John, can you confirm this? If so, that means that NANOOS would have to something "extra" to make our GeoServer WMS (and WFS?) endpoints nicely parseable by NGDC; that would be a bit disheartening, b/c it's extra work.

@pacioos https://github.com/pacioos, can you confirm that you're creating WAF ISO metadata records for your WMS/WFS endpoints/datasets? If so, can you tell us briefly how you generate these? Are they hand crafted, or do they involve some step that scrapes (eg, via an XSLT) information from the WMS & WFS GetCapabilities to pre-fill sections of the ISO metadata?

— Reply to this email directly or view it on GitHubhttps://github.com/ioos/registry/issues/19#issuecomment-43254680 .

amilan17 commented 10 years ago

Hi All, I'm looking at the XSL that generates ISO from WMS and I see that there some errors-- working on them now and will let you know when I think it is in better shape. Cheers,

On Thu, May 15, 2014 at 2:21 PM, Pacific Islands Ocean Observing System (PacIOOS) notifications@github.com wrote:

@amilan17 https://github.com/amilan17, have you had a chance to think about how the NANOOS WMS endpoint is being harvested into NGDC GeoPortal? Like I said, there's much more information in my WMS GetCapabilities that's not captured in the GeoPortal record, for example the spatial bounding box, contact information, etc.

Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

emiliom commented 10 years ago

@pacioos, thanks for chiming in and explaining what you did! Very helpful.

@amilan17, I look forward to seeing what XSL-fu you come up with! Thanks.

amilan17 commented 10 years ago

I just completed the update the WMS translation to ISO. I think it is much improved and we should be seeing these updates by next week.

I added display EX_GeographicBoundingBox IF it is described in the top/first layer, fixed errors with contact information and add CRS/EPSG codes to referenceSystemInfo.

Please review next week and let me know how it's working and/or if you have other suggestions.

Thanks for your feedback.

On Thu, May 15, 2014 at 1:32 PM, Emilio Mayorga notifications@github.comwrote:

@amilan17 https://github.com/amilan17, have you had a chance to think about how the NANOOS WMS endpoint is being harvested into NGDC GeoPortal? Like I said, there's much more information in my WMS GetCapabilities that's not captured in the GeoPortal record, for example the spatial bounding box, contact information, etc.

Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

emiliom commented 10 years ago

Awesome, thanks Anna! I look forward to seeing it next week.

On Fri, May 16, 2014 at 10:41 AM, Anna Milan notifications@github.comwrote:

I just completed the update the WMS translation to ISO. I think it is much improved and we should be seeing these updates by next week.

I added display EX_GeographicBoundingBox IF it is described in the top/first layer, fixed errors with contact information and add CRS/EPSG codes to referenceSystemInfo.

Please review next week and let me know how it's working and/or if you have other suggestions.

Thanks for your feedback.

On Thu, May 15, 2014 at 1:32 PM, Emilio Mayorga <notifications@github.com

wrote:

@amilan17 https://github.com/amilan17, have you had a chance to think about how the NANOOS WMS endpoint is being harvested into NGDC GeoPortal? Like I said, there's much more information in my WMS GetCapabilities that's not captured in the GeoPortal record, for example the spatial bounding box, contact information, etc.

Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

—

Reply to this email directly or view it on GitHubhttps://github.com/ioos/registry/issues/19#issuecomment-43359026 .

emiliom commented 10 years ago

@amilan17 have you made progress on this, with the XSL changes you were making? The record doesn't look changes on the NGDC GeoPortal. Thanks,

amilan17 commented 10 years ago

Emilio, How do they look in the NANOOS WAF? http://www.ngdc.noaa.gov/metadata/published/NOAA/IOOS/NANOOS/iso/

Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

On Fri, May 23, 2014 at 9:43 AM, Emilio Mayorga notifications@github.comwrote:

@amilan17 https://github.com/amilan17 have you made progress on this, with the XSL changes you were making? The record doesn't look changes on the NGDC GeoPortal. Thanks,

— Reply to this email directly or view it on GitHubhttps://github.com/ioos/registry/issues/19#issuecomment-44027364 .

emiliom commented 10 years ago

It looks better. The Point of Contact and extent / geographicElement are now there. But layers are not recognized and parsed.

I assume you haven't pushed this from the WAF to this GeoPortal "Details" page? http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={D4B1CAF9-1368-4CFF-BA1B-CF0B94FB0F3C} That page looks unchanged (and very sparse). Thanks.

emiliom commented 10 years ago

@amilan17, just wondering if there's some progress on this. Sorry to keep bugging; I know you have a million things on your plate, and WMS is probably not at the top. Thanks!

amilan17 commented 10 years ago

Hi Emilio, Thanks for the reminder. Answers are inline below.

On Tue, May 27, 2014 at 7:43 PM, Emilio Mayorga notifications@github.com wrote:

But layers are not recognized and parsed.

I currently do not have plans to update the WMS-to-ISO XSL to include the layers. This would be a great addition to the transform, but it will take some thought and trial/error to implement.

I assume you haven't pushed this from the WAF to this GeoPortal "Details" page? ...That page looks unchanged (and very sparse).

This record has been synced to Geoportal and I would not rely on the "Details" page to provide a very thorough view of the metadata. It is a standard out-of-box presentation and may not include everything that is actually in the XML file.

I recommend viewing the metadata from the WAF instead. This view will show everything that is in the file:

http://www.ngdc.noaa.gov/docucomp/page?xml=NOAA/IOOS/NANOOS/iso/xml/data.nanoos.org_geoserver_nvs_ows.xml&view=xml2text/xml-to-text-ISO

Sincerely, Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

amilan17 commented 10 years ago

Actually, You are right. It's been harvested to Geoportal but failing validation, because it doesn't like the dateStamp. This is a result of the transform and I will fix this tomorrow.

Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

On Thu, Jun 12, 2014 at 4:57 PM, Anna Milan - NOAA Federal < anna.milan@noaa.gov> wrote:

Hi Emilio, Thanks for the reminder. Answers are inline below.

On Tue, May 27, 2014 at 7:43 PM, Emilio Mayorga notifications@github.com wrote:

But layers are not recognized and parsed.

I currently do not have plans to update the WMS-to-ISO XSL to include the layers. This would be a great addition to the transform, but it will take some thought and trial/error to implement.

I assume you haven't pushed this from the WAF to this GeoPortal "Details" page? ...That page looks unchanged (and very sparse).

This record has been synced to Geoportal and I would not rely on the "Details" page to provide a very thorough view of the metadata. It is a standard out-of-box presentation and may not include everything that is actually in the XML file.

I recommend viewing the metadata from the WAF instead. This view will show everything that is in the file:

http://www.ngdc.noaa.gov/docucomp/page?xml=NOAA/IOOS/NANOOS/iso/xml/data.nanoos.org_geoserver_nvs_ows.xml&view=xml2text/xml-to-text-ISO

Sincerely, Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

emiliom commented 10 years ago

@amilan17, the Geoportal Details page at http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={D4B1CAF9-1368-4CFF-BA1B-CF0B94FB0F3C} still looks unchanged. Is the metadata still failing validation? Thanks!

amilan17 commented 10 years ago

Emilio, The record is there and passing validation. It was last synced this morning. It is still not translating the layers. And this might be a better view of the metadata record for review. http://www.ngdc.noaa.gov/docucomp/page?xml=NOAA/IOOS/NANOOS/iso/xml/data.nanoos.org_geoserver_nvs_ows.xml&view=xml2text/xml-to-text-ISO

Anna ~~~~~~~ Anna.Milan@noaa.gov, 303-497-5099 NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma ~~~~~~~

On Tue, Jun 17, 2014 at 9:10 AM, Emilio Mayorga notifications@github.com wrote:

@amilan17 https://github.com/amilan17, the Geoportal Details page at http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={D4B1CAF9-1368-4CFF-BA1B-CF0B94FB0F3C} still looks unchanged. Is the metadata still failing validation? Thanks!

— Reply to this email directly or view it on GitHub https://github.com/ioos/registry/issues/19#issuecomment-46320179.

emiliom commented 10 years ago

Good, thanks. I realize the EMMA URL you're pointing to is definitely more complete (better) and faithful to the iso XML. But the Geoportal document is likely to be more publicly exposed and accessible. So I guess I'm wondering why certain information that's present in the iso XML are not shown on the Geoportal Details page, like Organization and Organization Role, Contact Information, Spatial Domain. Those attributes are exposed in the NGDC Geoportal Details, in the only somewhat comparable IOOS WMS record I can find on the IOOS Catalog (from PacIOOS): http://www.ngdc.noaa.gov/geoportal/catalog/search/resource/details.page?uuid={3CB74631-5782-43D4-9924-F09EA11F0AEA} Except, as mentioned earlier on this thread, that WMS record is based on hand-crafted WAF iso XML, not your wms GetCapabilities xml > iso converter.

I'm also just trying to get to the bottom of what will actually be exposed and where from this NANOOS WMS endpoint. I guess I'm slowly coming to the conclusion that we may have to hand generate iso XML's for individual WMS layers (probably taking PacIOOS' template) if we want to expose and catalog these services in detail. Though the IOOS Catalog itself (http://catalog.ioos.us) doesn't yet parse WMS layer information in much detail at all; the rich PacIOOS record becomes this skimpy IOOS Catalog service record. But that's a different topic, more for @daf and IOOS Catalog discussions.

Thanks for all your help!!