HDFGroup / hsds

Cloud-native, service based access to HDF data
https://www.hdfgroup.org/solutions/hdf-kita/
Apache License 2.0
128 stars 52 forks source link

Conflicting CORS headers cause exception on binary data read for dataset/{id}/value GET request #96

Closed xaxis-ii closed 2 years ago

xaxis-ii commented 3 years ago

The following lines in chunk_sn.py are superfluous and conflict with aiohttp's internal (now enabled) CORS handling code if the Origin header is set on an incoming request. They should be removed as CORS is handled automatically without them.

1909            # allow CORS
1910            if cors_domain:
1911                resp.headers['Access-Control-Allow-Origin'] = cors_domain
1912                cors_methods = "GET, POST, DELETE, PUT, OPTIONS"
1913                resp.headers['Access-Control-Allow-Methods'] = cors_methods
1914                cors_headers = "Content-Type, api_key, Authorization"
1915                resp.headers['Access-Control-Allow-Headers'] = cors_headers

Resulting exception ('Error handling request') looks like this in docker logs:

REQ> GET: /datasets/d-e1fdb4c1-66c65542-0b07-cbdd76-969117/value [/modules/005F02Q]
INFO> getObjectJson d-e1fdb4c1-66c65542-0b07-cbdd76-969117
INFO> validateAction(domain=dev/modules/005F02Q, obj_id=d-e1fdb4c1-66c65542-0b07-cbdd76-969117, username=admin, action=read)
INFO> getDomainJson(dev/modules/005F02Q, reload=False)
INFO> aclCheck: read for user: admin
INFO> doHyperSlabRead - number of chunk_ids: 1
INFO> read_chunk_hyperslab, chunk_id: c-e1fdb4c1-66c65542-0b07-cbdd76-969117_0, slices: (slice(0, 1, 1),), bucket: dev, serverless: False
INFO> chunk_arr shape: (1,)
INFO> data_sel: (slice(0, 1, 1),)
ERROR> Exception during binary data write: 
Error handling request
jreadey commented 3 years ago

Yes, original I was handling CORS manually and then removed the manual steps once the internal cors handling was available. So I can believe I missed taking this out for the /value request.

I wasn't able to reproduce the error by adding the Origin header. See: https://github.com/HDFGroup/hsds/blob/lambda2/tests/integ/value_test.py#L1443. Am I missing something?

xaxis-ii commented 3 years ago

The issue only occurs when downloading with the Accept: Octet-Stream type set to request binary return value. The test you've linked doesn't set this header and downloads json, which bypasses the problematics lines within the binary response branch of the if statement.

jreadey commented 2 years ago

Ok - that's helpful - I can actually see the fail now. Checked in the fix to the lambda2 branch. Will merge with master soon.

jreadey commented 2 years ago

This is checked in in master now. Please re-open if this is still an issue.