search

OSGeo / PROJ-data

Repository for proj datum grids (for use by PROJ 7 or later)
Other
70 stars 32 forks source link

https://cdn.proj.org response headers in case of CORS #95

Closed aharondavid closed 1 year ago

aharondavid commented 1 year ago

PROJ read grid shift, by default , from https://cdn.proj.org. when downloading from this URL, the AWS server set the Access-Control-Allow-Origin to '*' but, for reading the 'Range' header in case of CORS, the 'Access-Control-Expose-Headers' should also be set to '*'

rouault commented 1 year ago

@aharondavid S3 doesn't allow the wildcard value "*". So I've just added "Range". CC @hobu the new CORS configuration is thus

[
    {
        "AllowedHeaders": [
            "Authorization"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [
            "Range"
        ],
        "MaxAgeSeconds": 3000
    }
]
aharondavid commented 1 year ago

Actually, to be more precisely, the headers that used by the functionproj_network_get_header_value_cbk_typeare:

  1. 'Content-Range'
  2. 'Last-Modified'
  3. 'ETag'

Can you add them too

rouault commented 1 year ago

Can you add them too

done

aharondavid commented 1 year ago

Thanks!

aharondavid commented 1 year ago

I think, there is a regression. now I get from the browser the following error: Access to XMLHttpRequest at 'https://cdn.proj.org/us_nga_egm96_15.tif' from origin 'http://localhost:6931' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

hobu commented 1 year ago

I don't understand why proj_network_get_header_value_cbk_type needs any CORS headers set at all. Your running inside of PROJ, right? This isn't a web page.

Please post specific errors with specific code examples. We've been operating the CDN for 2+ years without any issues, and I think this is all a red herring. We need to be proven otherwise before we should go twiddling stuff that could break things for a lot of folks.

aharondavid commented 1 year ago

proj_network_get_header_value_cbk_type is callback that go out to html\js that call to https://cdn.proj.org/us_nga_egm96_15.tif request, here is the error from the DevTool: image

hobu commented 1 year ago

I built the PROJ by Emscripten to WebAssembly

OK, this makes more sense. Is there any opportunity for you to contribute back your Emscripten build configuration and usages?

It's a bit dangerous for us to be changing headers on the live production CDN. Especially the ETag one, I think.

rouault commented 1 year ago

I've added "Access-Control-Allow-Origin" to the list, but obviously I hardly understand all that stuff.

@hobu Should I just revert all the changes done in ExposeHeaders and restore it to [] ?

aharondavid commented 1 year ago

here are an html\js full code example: you can run it in cmd: node server.js and you will see the CORS error.

server.js:

const http = require('http');
const fs = require('fs');

const port = 3000;

http.createServer((req, res) => {

  const filePath = './testCORS.html';

  fs.readFile(filePath, (err, data) => {
    if (err) {
      res.writeHead(500);
      res.end(`Error: ${err}`);
      return;
    }

    res.writeHead(200, {
      'Content-Type': 'text/html',
      'Access-Control-Allow-Origin': '*'
    });
    res.end(data);
  });
}).listen(port, () => {
  console.log(`Server running on port ${port}`);
});

testCORS.html:

<html>
<head>
  <title>XMLHttpRequest Example</title>
</head>
<body onload="makeRequest()">

  <script>
    function makeRequest() {    
      const xhr = new XMLHttpRequest();
      xhr.open('GET', 'https://cdn.proj.org/us_nga_egm96_15.tif');
     //xhr.open('GET', 'https://dev.skylinesoft.com/sg/aharon/us_nga_egm96_15.tif'); this an example of server that is work.
      xhr.setRequestHeader('Range', 'bytes=100-199');
      xhr.onreadystatechange = function() {
        if (this.readyState === 4) {
          console.log(`Response status code: ${this.status}`);
          console.log('Response headers:');
          console.log(this.getAllResponseHeaders());
          console.log(`Response body: ${this.responseText}`);
        }
      };

      xhr.send();
    }
  </script>

</body>
</html>
connormanning commented 1 year ago

Your error is coming from the preflight OPTIONS request - not the GET itself. And the reason for that failure is that the OPTIONS request is coming back as 403 forbidden:

options

So this is not about GET parameters but rather that the preflight is failing so the browser auto-fails the GET. Maybe OPTIONS needs to be added to AllowedMethods?

connormanning commented 1 year ago

It looks like this is from CloudFront rather than S3 directly (right?). @rouault perhaps make sure that you have GET/HEAD/OPTIONS allowed here (from the "Behaviors" tab):

cloudfront
rouault commented 1 year ago

I've reverted the CORS configuration of the bucket back to its initial state:

[
    {
        "AllowedHeaders": [
            "Authorization"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]

and I've changed the CloudFront configuration to @connormanning 's above suggestion

aharondavid commented 1 year ago

Still get a CORS error

connormanning commented 1 year ago

CloudFront changes take 15+ minutes to deploy - give it another 20 or 30 minutes.

hobu commented 1 year ago

Configuration seems to be working for me now. Please confirm and we can close.

connormanning commented 1 year ago

Works for me now.

aharondavid commented 1 year ago

Hi, sorry about delay (I just now come back from weekend).

Now, there is no CORS but the above headers are missing from the response:

  1. 'Content-Range'
  2. 'Last-Modified'
  3. 'ETag'

They are crucial for proj_network_get_header_value_cbk_type As I wrote above, I think that the Access-Control-Expose-Headers should be set to '*' or explicitly add those headers