search

WikiWatershed / model-my-watershed

The web application front end for Model My Watershed.
https://modelmywatershed.org
Apache License 2.0
57 stars 31 forks source link

CINERGI CSW 3.0 API Documentation #1929

Closed kdeloach closed 7 years ago

kdeloach commented 7 years ago

There are significant differences between results from the CINERGI CSW 3.0 API which impacts our ability to integrate with this service. We need the documentation for this service to know which data format to expect from the API.

Here are some differences I noticed between items (from the same request) while testing the API:

Sample query

http://132.249.238.169:8080/geoportal/opensearch?q=philadelphia&f=json&bbox=-80.06029129028319%2C40.304403558775284%2C-79.94699478149414%2C40.42813291388417

Sample responses

Item 1 ```json { "_index":"metadata_v5", "_type":"item", "_id":"dd1fea01f0a64e8ba3bf93308870b7ca", "_score":14.328026, "_source":{ "src_source_type_s":"WAF", "src_source_uri_s":"WAF:http://132.249.238.151:8080/cinergi/waf/ScienceBase_WAF/", "src_source_name_s":"ScienceBase_processed", "src_uri_s":"http://132.249.238.151:8080/cinergi/waf/ScienceBase_WAF/0125/52167a76e4b0b45d6ba58d7a.xml", "src_lastupdate_dt":"2017-01-16T20:37:48Z", "sys_created_dt":"2017-03-17T22:28:07.080Z", "sys_modified_dt":"2017-03-17T22:28:07.080Z", "sys_xmlmodified_dt":"2017-03-17T22:28:07.080Z", "sys_owner_s":"gptadmin", "sys_metadatatype_s":"iso19115-2", "fileid":"52167a76e4b0b45d6ba58d7a", "title":"USGS 1:24000-scale Quadrangle for Philadelphia, MO 1949", "description":"USGS Historical Quadrangle in GeoPDF.", "keywords_s":[ "imageryBaseMapsEarthCover", "Downloadable Data", "Map", "Historical Topographic Maps 7.5 x 7.5 Minute Map Series", "Historical Topographic Maps", "7.5 x 7.5 minute", "GeoPDF", "MO", "Philadelphia", "Topographic", "USGS", "Topographic Map", "Topographic", "MAPS", "Topographic", "geoscientificInformation" ], "links_s":[ "https://www.sciencebase.gov/catalog/item/52167a76e4b0b45d6ba58d7a", "http://ims.er.usgs.gov/gda_services/download?item_id=5579892", "https://www.sciencebase.gov/catalog/file/get/52167a76e4b0b45d6ba58d7a" ], "contact_organizations_s":[ "U.S. Geological Survey", "U.S. Geological Survey", "U.S. Geological Survey", "U.S. Geological Survey" ], "apiso_Identifier_s":"52167a76e4b0b45d6ba58d7a", "apiso_Title_txt":"USGS 1:24000-scale Quadrangle for Philadelphia, MO 1949", "apiso_Abstract_txt":"USGS Historical Quadrangle in GeoPDF.", "apiso_OrganizationName_txt":"U.S. Geological Survey", "apiso_Subject_txt":[ "imageryBaseMapsEarthCover", "Downloadable Data", "Map", "Historical Topographic Maps 7.5 x 7.5 Minute Map Series", "Historical Topographic Maps", "7.5 x 7.5 minute", "GeoPDF", "MO", "Philadelphia", "Topographic", "USGS", "Topographic Map", "Topographic", "MAPS", "Topographic", "geoscientificInformation" ], "apiso_TopicCategory_s":"geoscientificInformation", "apiso_KeywordType_s":[ "Theme", "Theme", "Theme", "Theme", "Theme", "Theme", "Place", "theme", "theme", "theme", "theme", "theme", "theme" ], "apiso_Type_s":"dataset", "apiso_Modified_dt":"2014-08-20T15:04:30Z", "apiso_Language_s":"eng: USA", "apiso_HasSecurityConstraints_b":false, "envelope_geo":{ "type":"envelope", "coordinates":[ [ -91.75, 39.875 ], [ -91.625, 39.75 ] ] }, "envelope_cen_pt":{ "lat":39.8125, "lon":-91.6875 }, "categories_cat":[ "Resource Type > Document > Topographic", "Organization > GOVERNMENT AGENCIES-U.S. FEDERAL AGENCIES > USGS", "Resource Type > Spatial Data > Topographic Map", "Property > Measure > Topographic", "Equipment > Instrument > MAPS", "Feature > Physiographic Feature > Topographic" ], "hierarchies_cat":[ "Category > Feature > Physiographic Feature > Topographic", "Category > Equipment > Instrument > Earth Remote Sensing Instruments > Passive Remote Sensing > Spectrometers/Radiometers > Spectrometers > Measurement of Air Pollution from Satellite", "Category > Property > Measure > Topographic", "Category > Resource Type > Spatial Data > Topographic Maps", "Category > Organization > GOVERNMENT AGENCIES-U.S. FEDERAL AGENCIES > USGS", "Category > Resource Type > Document > Topographic" ] } }, ```
Item 2 ```json { "_index":"metadata_v5", "_type":"item", "_id":"cde34521ebf044a09af2ea9b4a463e92", "_score":15.219486, "_source":{ "src_source_type_s":"WAF", "src_source_uri_s":"WAF:http://132.249.238.151:8080/cinergi/waf/ScienceBase_WAF/", "src_source_name_s":"ScienceBase_processed", "src_uri_s":"http://132.249.238.151:8080/cinergi/waf/ScienceBase_WAF/0006/528a8458e4b07c3230e6a4c6.xml", "src_lastupdate_dt":"2017-01-14T08:02:26Z", "sys_created_dt":"2017-03-17T17:30:16.025Z", "sys_modified_dt":"2017-03-17T17:30:16.025Z", "sys_xmlmodified_dt":"2017-03-17T17:30:16.025Z", "sys_owner_s":"gptadmin", "sys_metadatatype_s":"iso19115-2", "fileid":"528a8458e4b07c3230e6a4c6", "title":"ESMERALDA (AURORA) MINING DISTRICT, MINERAL COUNTY, BLM PATENTED CLAIMS", "description":"APPROXIMATELY LOCATED IN OR AROUND THE AURA MINING DISTRICT; LIST OF U.S. BUREAU OF MINES PATENTED CLAIM RECORDS; ESMERALDA (AURORA) MINING DISTRICT; SITE NAME: RADICAL; SPOTTED TIGER; BALD EAGLE; PHILADELPHIA; PHILADELPHIA NO. 2; LAST CHANCE; JUNIATA; ANTELOPE; CORTEZ; EAST GARIBALDI; LADY JANE; UTAH; SONORA; GOLDEN AGE; EMPIRE; ESMERALDA; LIVE YANKEE; DURAND; PROSPECTUS; NORTHERN BELLE; AND MORE; AUTHOR: U.S. BUREAU OF MINES", "keywords_s":[ "Paper Records; TIF; PDF", "AURA", "LIST", "geoscientificInformation" ], "links_s":"https://www.sciencebase.gov/catalog/item/528a8458e4b07c3230e6a4c6", "contact_organizations_s":"Unknown", "apiso_Identifier_s":"528a8458e4b07c3230e6a4c6", "apiso_Title_txt":"ESMERALDA (AURORA) MINING DISTRICT, MINERAL COUNTY, BLM PATENTED CLAIMS", "apiso_AlternateTitle_txt":"P687; 4100063", "apiso_Abstract_txt":"APPROXIMATELY LOCATED IN OR AROUND THE AURA MINING DISTRICT; LIST OF U.S. BUREAU OF MINES PATENTED CLAIM RECORDS; ESMERALDA (AURORA) MINING DISTRICT; SITE NAME: RADICAL; SPOTTED TIGER; BALD EAGLE; PHILADELPHIA; PHILADELPHIA NO. 2; LAST CHANCE; JUNIATA; ANTELOPE; CORTEZ; EAST GARIBALDI; LADY JANE; UTAH; SONORA; GOLDEN AGE; EMPIRE; ESMERALDA; LIVE YANKEE; DURAND; PROSPECTUS; NORTHERN BELLE; AND MORE; AUTHOR: U.S. BUREAU OF MINES", "apiso_OrganizationName_txt":"Unknown", "apiso_Subject_txt":[ "Paper Records; TIF; PDF", "AURA", "LIST", "geoscientificInformation" ], "apiso_TopicCategory_s":"geoscientificInformation", "apiso_KeywordType_s":"theme", "apiso_Type_s":"dataset", "apiso_Modified_dt":"2013-11-18T21:19:20Z", "apiso_Language_s":"eng: USA", "apiso_HasSecurityConstraints_b":false, "envelope_geo":[ { "type":"envelope", "coordinates":[ [ -92.259631, 41.515837 ], [ -92.219631, 41.475837 ] ] }, { "type":"envelope", "coordinates":[ [ -75.31279, 40.101339 ], [ -75.01479, 39.803339 ] ] }, { "type":"envelope", "coordinates":[ [ 29.932349, 31.222538 ], [ 29.942349, 31.212538 ] ] } ], "envelope_cen_pt":[ { "lat":41.495836999999995, "lon":-92.239631 }, { "lat":39.952339, "lon":-75.16379 }, { "lat":31.217537999999998, "lon":29.937348999999998 } ], "categories_cat":[ "Equipment > Satellites > AURA", "Equipment > Satellites > LIST" ], "hierarchies_cat":[ "Category > Equipment > Satellites > Earth Observation Satellites > AURA", "Category > Equipment > Satellites > Earth Observation Satellites > NASA Decadal Survey > LIST" ] } }, ```
Item 3 ```json { "_index":"metadata_v5", "_type":"item", "_id":"05ebdd73876849119479de8063c82dff", "_score":6.1364346, "_source":{ "src_source_type_s":"WAF", "src_source_uri_s":"WAF:http://132.249.238.151:8080/cinergi/waf/Data.gov/", "src_source_name_s":"Data.Gov", "src_uri_s":"http://132.249.238.151:8080/cinergi/waf/Data.gov/0070/1910acdf-06d7-4e55-a7cb-2e27f166a05a.xml", "src_lastupdate_dt":"2016-11-14T01:02:43Z", "sys_created_dt":"2017-03-15T07:46:56.212Z", "sys_modified_dt":"2017-03-15T07:46:56.212Z", "sys_xmlmodified_dt":"2017-03-15T07:46:56.212Z", "sys_owner_s":"gptadmin", "sys_metadatatype_s":"iso19115-2", "fileid":"1910acdf-06d7-4e55-a7cb-2e27f166a05a", "title":"Monthly and Annual Mean Seawater Temperature, Salinity, and Density Data from 26 USA Tide Gauge Sites during 1855-1993", "description":"Tidal observers at primary tide gauges of the United StatesCoast and Geodetic Survey (now the NOAA National OceanService) routinely measured seawater temperature anddensity throughout most of the 20th century with selectsites well before. All available records for 26 siteshave been digitized, although more paper file recordsmay still exist for others. Students at Florida Instituteof Technology provided the key entry and quality control.", "keywords_s":[ "seawater temperature", "salinity", "density", "USA Tide Stations", "Continental United States coasts", "Western United States coasts", "Eastern United States coasts", "United States Gulf coast", "United States Alaskan coast", "Eastport, Maine", "Portland, Maine", "Boston, Massachusettes", "New York (The Battery), New York", "Sandy Hook, New Jersey", "Atlantic City, New Jersey", "Breakwater, Deleware", "Baltimore, Maryland", "Philadelphia, Pennsylvania", "Charleston, South Carolina", "Daytona Beach, Florida", "Key West, Florida", "Cedar Keys, Florida", "Pensacola, Florida", "Galveston, Texas", "San Francisco, California", "La Jolla, California", "Los Angeles Harbor, California", "Astoria, Oregon", "Port Townsend, Washington", "Seattle, Washington", "Sitka, Alaska", "Seward, Alaska", "Ketchikan, Alaska", "Kodiak Island (Women's Bay) Alaska", "Survey", "NOAA", "Island", "Temperature", "Salinity", "Density", "Gauge", "Continental", "Coast", "Harbor", "Gulf", "Tidal", "National Oceanographic Data Center", "Beach" ], "links_s":"http://www.nodc.noaa.gov/cgi-bin/search/prod/accessionsView.pl/details/0000817", "contact_organizations_s":[ "NOAA/NESDIS/NODC/NCDDC", "Dr. George A. Maul", "Florida Institute of Technology", "Andia M. Davis", "Florida Institute of Technology", "Jeffrey W. Simmons", "Bahamas Department of Meteorology", "NOAA/National Oceanographic Data Center", "NOAA/NESDIS/National Oceanographic Data Center", "Florida Institute of Technology", "A. Carvalho", "G. A. Maul", "K. Hanson", "G. A Maul", "United Nations Environment Programme", "G. A. Maul", "A. M. Davis", "D. Roemmich", "United States Government Printing Office", "U.S. Coast and Geodetic Survey", "U.S. National Ocean Service", "NOAA/NESDIS/NODC/NCDDC" ], "contact_people_s":[ "Mr. Patrick C. Caldwell", "Data Access Group, User Services Team", "George A. Maul", "Mr. Patrick C. Caldwell" ], "apiso_Identifier_s":"1910acdf-06d7-4e55-a7cb-2e27f166a05a", "apiso_Title_txt":"Monthly and Annual Mean Seawater Temperature, Salinity, and Density Data from 26 USA Tide Gauge Sites during 1855-1993", "apiso_Abstract_txt":"Tidal observers at primary tide gauges of the United StatesCoast and Geodetic Survey (now the NOAA National OceanService) routinely measured seawater temperature anddensity throughout most of the 20th century with selectsites well before. All available records for 26 siteshave been digitized, although more paper file recordsmay still exist for others. Students at Florida Instituteof Technology provided the key entry and quality control.", "apiso_OrganizationName_txt":"NOAA/NESDIS/NODC/NCDDC", "apiso_Subject_txt":[ "seawater temperature", "salinity", "density", "USA Tide Stations", "Continental United States coasts", "Western United States coasts", "Eastern United States coasts", "United States Gulf coast", "United States Alaskan coast", "Eastport, Maine", "Portland, Maine", "Boston, Massachusettes", "New York (The Battery), New York", "Sandy Hook, New Jersey", "Atlantic City, New Jersey", "Breakwater, Deleware", "Baltimore, Maryland", "Philadelphia, Pennsylvania", "Charleston, South Carolina", "Daytona Beach, Florida", "Key West, Florida", "Cedar Keys, Florida", "Pensacola, Florida", "Galveston, Texas", "San Francisco, California", "La Jolla, California", "Los Angeles Harbor, California", "Astoria, Oregon", "Port Townsend, Washington", "Seattle, Washington", "Sitka, Alaska", "Seward, Alaska", "Ketchikan, Alaska", "Kodiak Island (Women's Bay) Alaska", "Survey", "NOAA", "Island", "Temperature", "Salinity", "Density", "Gauge", "Continental", "Coast", "Harbor", "Gulf", "Tidal", "National Oceanographic Data Center", "Beach" ], "apiso_KeywordType_s":[ "theme", "place", "theme", "theme", "theme", "theme", "theme", "theme", "theme", "theme", "theme", "theme" ], "apiso_Type_s":"dataset", "apiso_Modified_dt":"2002-11-14T00:00:00Z", "apiso_AccessConstraints_s":"NOAA makes no warranty regarding these data, expressed or implied, nor does the fact of distribution constitute such a warranty. NOAA and NODC cannot assume liability for any damages caused by any errors or omissions in these data, nor as a result of the failure of these data to function on a particular system.", "apiso_OtherConstraints_s":[ "Use Constraints: NOAA and NODC would appreciate recognition as the resource from which these data were obtained in any publications and/or other representations of these data.", "Access Constraints: None" ], "apiso_HasSecurityConstraints_b":true, "services_nst":{ "url_s":"http://www.nodc.noaa.gov/cgi-bin/search/prod/accessionsView.pl/details/0000817", "url_type_s":"ASCII", "url_name_s":"ASCII" }, "envelope_geo":{ "type":"envelope", "coordinates":[ [ -152.52, 60.12 ], [ -66.99, 24.9 ] ] }, "envelope_cen_pt":{ "lat":42.51, "lon":-109.755 }, "timeperiod_nst":[ { "begin_dt":"1997-01-01T00:00:00Z", "begin_indeterminate_s":null, "end_dt":"1997-01-01T23:59:59.999Z", "end_indeterminate_s":null }, { "begin_dt":"1993-01-01T00:00:00Z", "begin_indeterminate_s":null, "end_dt":"1993-01-01T23:59:59.999Z", "end_indeterminate_s":null }, { "begin_dt":"2001-01-01T00:00:00Z", "begin_indeterminate_s":null, "end_dt":"2001-01-01T23:59:59.999Z", "end_indeterminate_s":null }, { "begin_dt":"1992-01-01T00:00:00Z", "begin_indeterminate_s":null, "end_dt":"1992-01-01T23:59:59.999Z", "end_indeterminate_s":null }, { "begin_dt":"1929-01-01T00:00:00Z", "begin_indeterminate_s":null, "end_dt":"1929-01-01T23:59:59.999Z", "end_indeterminate_s":null }, { "begin_dt":"1983-01-01T00:00:00Z", "begin_indeterminate_s":null, "end_dt":"1983-01-01T23:59:59.999Z", "end_indeterminate_s":null } ], "apiso_ConditionApplyingToAccessAndUse_txt":[ "Use Constraints: NOAA and NODC would appreciate recognition as the resource from which these data were obtained in any publications and/or other representations of these data.", "Access Constraints: None" ], "apiso_ResponsiblePartyRole_txt":"pointOfContact", "categories_cat":[ "Activity > Observation > Survey", "Organization > GOVERNMENT AGENCIES-U.S. FEDERAL AGENCIES > NOAA", "Feature > Hydrologic Feature > Island", "Property > Measure > Temperature", "Property > Measure > Salinity", "Property > Measure > Density", "Equipment > Instrument > Gauge", "Realm > Continental > Continental", "Feature > Marine Feature > Coast", "Feature > Marine Feature > Harbor", "Feature > Marine Feature > Gulf", "Realm > Coastal Region > Tidal", "Feature > Physiographic Feature > Beach" ] } } ```
kdeloach commented 7 years ago

It looks like the varying response formats are a result of customizations made to Geoportal Server to support CINERGI data sources:

https://github.com/cinergi/geoportal-server-catalog

aufdenkampe commented 7 years ago

David Valentine (@valentinedwv), Tom Whitenack (@tom2275) or Ilya Zaslavsky (@izaslavsky), can you help Azavea with these questions?

valentinedwv commented 7 years ago

We can customize and reindex. It's an index. if a field is not present, then it will not be there.

links_s field may be a string or list of strings

Links have no context (is it a details URL? is it a download URL? etc.)

Unclear which field to use for source URL (0 or more fields may be present): src_source_uri_s src_uri_s links_s services_nst.url_s

envelope_geo field may be absent, an object, or a list of objects

rajadain commented 7 years ago

Commenting to subscribe to this thread and get email notifications.

valentinedwv commented 7 years ago

https://github.com/Esri/geoportal-server-catalog/blob/master/geoportal/doc/api.txt Single Record (need to improve pretty itemcard like html?) http://cinergi.sdsc.edu/geoportal/csw?service=CSW&request=GetRecordById&id=58ac0e4d0f714cb5b964dae61d14ce2f

http://132.249.238.169:8080/geoportal/rest/metadata/item/58ac0e4d0f714cb5b964dae61d14ce2f/html

ajrobbins commented 7 years ago

Question has been resolved.