search

ioos / ckanext-ioos-theme

IOOS Catalog as a CKAN extension
GNU Affero General Public License v3.0
7 stars 14 forks source link

Muliple Access-Control_Allow-Origin headers: CORS enabled twice? #230

Closed rsignell-usgs closed 4 years ago

rsignell-usgs commented 4 years ago

I'm trying to hit the IOOS CKAN Endpoint with TerriaJS and when I go to "add data" and click on "IOOS" to do the CKAN request, it's complaining about getting multiple Access-Control-Allow-Origin and that only one is allowed.

Here's a screengrab that shows the two of them:

2020-08-19_15-24-33

Is CORS being set on NGINX (or Apache) as well as on the Service itself or something?

rsignell-usgs commented 4 years ago

@kknee or @mwengren , who would be the right person to ping on this?
I'm guessing this should take 15 min to diagnose and fix?

mwengren commented 4 years ago

@rsignell-usgs Good to see you back on the IOOS Catalog airwaves!

@benjwadams would be the right person to track this down. We just made some hosting changes; perhaps it's related.

You should get faster response times from Catalog as a result, if the CORS issue can be resolved.

rsignell-usgs commented 4 years ago

@kknee , could someone else perhaps could take a look if @benjwadams is not available? This should be a quick fix.

benjwadams commented 4 years ago

@rsignell-usgs This looks like it was an issue on the Nginx web server side. I checked the CORS settings that data.ioos.us, and they were only imported/sourced once. I'm not sure what was causing the redundant CORS headers, but it appears that a "graceful" restart of the webserver seems to have been sufficient on my end to remove the extra headers. Please check on your end after clearing any cached versions of the web page.

rsignell-usgs commented 4 years ago

@benjwadams, I don't see any change on my end. I'm still getting the two Access-Control-Allow-Origin headers.

Can you test my use case?

  1. Go to https://data.amerigeoss.org/ameriterria/#clean&proxy/_60s/https://raw.githubusercontent.com/rsignell-usgs/ameriterria/master/catalogs/ioos.json
  2. Click on "add data"
  3. Click on "ioos".

If that works for you, I guess it must be something cached on my end.

zdefne-usgs commented 4 years ago

I get the same error too. Seems to be a CORS issue? CORS

benjwadams commented 4 years ago

@zdefne-usgs, @rsignell-usgs ,

Try using a utility such as cURL that doesn't attempt to cache requests and see what you get when requesting from the server.

Here are the results I have from issuing a GET request to the IOOS Catalog Homepage:

[badams@localhost]~% curl -s -i https://data.ioos.us | head -n20 
HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Wed, 26 Aug 2020 21:10:30 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 16609
Connection: keep-alive
Cache-Control: private
Set-cookie: ckan=1f050331536d5bb858a44fc6a8d3295e45b2126d786376061c2a44908b4b1df013b47f35; Path=/
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range
Access-Control-Expose-Headers: Content-Length,Content-Range

<!DOCTYPE html>
<!--[if IE 9]> <html lang="en" class="ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html lang="en"> <!--<![endif]-->
  <head>
    <link rel="stylesheet" type="text/css" href="/fanstatic/vendor/:version:2020-07-24T15:06:20/select2/select2.css" />
<link rel="stylesheet" type="text/css" href="/fanstatic/css/:version:2020-07-24T15:06:20/main.min.css" />
<link rel="stylesheet" type="text/css" href="/fanstatic/vendor/:version:2020-07-24T15:06:20/font-awesome/css/font-awesome.min.css" />
benjwadams commented 4 years ago

I just tried to add the catalog to that viewer. I looked at the JavaScript console and it seems to indicate a CORS issue with this URL: https://data.ioos.us/api/3/action/package_search?rows=100000&sort=metadata_created%20asc&fq=%2B(res_format%3A(geojson%20OR%20GeoJSON%20OR%20kml%20OR%20KML%20OR%20kmz%20OR%20KMZ%20OR%20wms%20OR%20WMS%20OR%20%22Esri%20REST%22))&start=0

However, cURLing this URL and inspecting the headers does not reveal that the CORS headers are sent twice anymore:

curl -i 'https://data.ioos.us/api/3/action/package_search?rows=100000&sort=metadata_created%20asc&fq=%2B(res_format%3A(geojson%20OR%20GeoJSON%20OR%20kml%20OR%20KML%20OR%20kmz%20OR%20KMZ%20OR%20wms%20OR%20WMS%20OR%20%22Esri%20REST%22))&start=0'

rsignell-usgs commented 4 years ago

@benjwadams , yes, on the one hand, the curl request seems to only return one:

$ curl -s -i 'https://data.ioos.us/api/3/action/package_search?rows=100000&sort=metadata_created%20asc&fq=%2B(res_format%3A(geojson%20OR%20GeoJSON%20OR%20kml%20OR%20KML%20OR%20kmz%20OR%20KMZ%20OR%20wms%20OR%20WMS%20OR%20%22Esri%20REST%22))&start=0' | head -c 900
HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Wed, 26 Aug 2020 21:49:14 GMT
Content-Type: application/json;charset=utf-8
Content-Length: 21769056
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range
Access-Control-Expose-Headers: Content-Length,Content-Range

{"help": "https://data.ioos.us/api/3/action/help_show?name=package_search", "success": true, "result": {"count": 1140, "sort": "metadata_created asc", "facets": {}, "results": [{"license_title" 
...

However, the TerriaJS developer tools show some different response returning before the response shown in the curl response:

HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Wed, 26 Aug 2020 21:45:32 GMT
Content-Type: application/json;charset=utf-8
Content-Length: 21769056
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, PUT, GET, DELETE, OPTIONS
Access-Control-Allow-Headers: X-CKAN-API-KEY, Authorization, Content-Type
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range
Access-Control-Expose-Headers: Content-Length,Content-Range

Note 

Access-Control-Allow-Methods: POST, PUT, GET, DELETE, OPTIONS

section is totally missing from the curl response.

So I don't think the curl command is mimicking the TerriaJS request exactly.

Do we need to engage a web developer?

benjwadams commented 4 years ago

OK, it's definitely showing browsable datasets right now, but it doesn't seem to be hitting WMS endpoints.

The solution was to remove the CORS header includes from Nginx for pages relating to CKAN -- I think these were included for CSW compatibility, but the CKAN application and API have their own set of headers which are supposed to be enabled by default in newer versions of CKAN. I still don't happen to know why headers are showing up in the TerriaJS request, but not cURL.

rsignell-usgs commented 4 years ago

@benjwadams thanks so much! Agreed, I don't know why the WMS endpoints aren't showing up, but that could be an issue with the CKAN API request, since I know exactly 0 about them.

But the main thing is, YOU FIXED THIS ISSUE!!!! 🍾