HDFGroup / hsds

Cloud-native, service based access to HDF data
https://www.hdfgroup.org/solutions/hdf-kita/
Apache License 2.0
129 stars 53 forks source link

AttributeError on invoking lambda #68

Closed jananzhu closed 1 year ago

jananzhu commented 3 years ago

Hi, we're testing out the lambda submission feature of HSDS and we're getting 500 errors on our read requests. I checked the SN logs and confirmed that it's trying to invoke the lambda but is running into the following AttributeError. A quick search for the error suggests that it may be related to aiobotocore?

ERROR> Unexpected exception for lamdea invoke: 'ClientCreatorContext' object has no attribute 'invoke', type: <class 'AttributeError'>
INFO> read_chunk_hyperslab, chunk_id: c-338aa60e-7a07b72a-80c1-34d2e1-85921c_249_0_0, slices: (slice(0, 500, 1), slice(2, 3, 1), slice(0, 3, 1)), bucket: redesign-hsds-prod, serverless: True
INFO> invoking lambda function chunk_read with payload: {'select': '[0:2,2:3,0:3]', 'bucket': 'redesign-hsds-prod', 'chunk_id': 'c-338aa60e-7a07b72a-80c1-34d2e1-85921c_249_0_0', 'dset_json': {'id': 'd-338aa60e-7a07b72a-80c1-34d2e1-85921c', 'root': 'g-338aa60e-7a07b72a-bb02-2e86f2-8f3fa2', 'created': 1602012530, 'lastModified': 1602012530, 'type': {'class': 'H5T_FLOAT', 'base': 'H5T_IEEE_F32LE'}, 'shape': {'class': 'H5S_SIMPLE', 'dims': [500, 46576, 3], 'maxdims': [0, 46576, 3]}, 'creationProperties': {'layout': {'class': 'H5D_CHUNKED', 'dims': [2, 46576, 3]}, 'fillTime': 'H5D_FILL_TIME_ALLOC', 'filters': [{'class': 'H5Z_FILTER_SHUFFLE', 'id': 2, 'name': 'shuffle'}, {'class': 'H5Z_FILTER_DEFLATE', 'id': 1, 'level': 1, 'name': 'gzip'}]}, 'layout': {'class': 'H5D_CHUNKED', 'dims': [2, 46576, 3]}, 'attributes': {'CLASS': {'type': {'class': 'H5T_STRING', 'charSet': 'H5T_CSET_ASCII', 'length': 6, 'strPad': 'H5T_STR_NULLPAD'}, 'shape': {'class': 'H5S_SCALAR'}, 'value': 'EARRAY', 'created': 1602012532.4391222}, 'EXTDIM': {'type': {'class': 'H5T_INTEGER', 'base': 'H5T_STD_I32LE'}, 'shape': {'class': 'H5S_SCALAR'}, 'value': 0, 'created': 1602012532.4724793}, 'TITLE': {'type': {'class': 'H5T_STRING', 'charSet': 'H5T_CSET_ASCII', 'length': 1, 'strPad': 'H5T_STR_NULLPAD'}, 'shape': {'class': 'H5S_SCALAR'}, 'value': 'E', 'created': 1602012532.5056005}, 'VERSION': {'type': {'class': 'H5T_STRING', 'charSet': 'H5T_CSET_ASCII', 'length': 3, 'strPad': 'H5T_STR_NULLPAD'}, 'shape': {'class': 'H5S_SCALAR'}, 'value': '1.1', 'created': 1602012532.5390193}, 'units': {'type': {'class': 'H5T_STRING', 'charSet': 'H5T_CSET_ASCII', 'length': 10, 'strPad': 'H5T_STR_NULLPAD'}, 'shape': {'class': 'H5S_SCALAR'}, 'value': 'nanometers', 'created': 1602012532.5726192}}}} start: 1602018839.911248
jreadey commented 1 year ago

This is not relevant anymore - the current code is not invoking Lambda functions from within the SN node, but as a complete HSDS implementation. See: https://github.com/HDFGroup/hsds/blob/master/docs/aws_lambda_setup.md.