NYCPlanning / data-engineering-qaqc

streamlit app for data engineering
https://edm-data-engineering.nycplanningdigital.com
1 stars 0 forks source link

fix error from new pluto_corrections zip name and one less file #251

Closed damonmcc closed 1 year ago

damonmcc commented 1 year ago

QAQC error with new data generated by https://github.com/NYCPlanning/db-pluto/pull/425:

NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: Unknown

Traceback:
File "/root/.cache/pypoetry/virtualenvs/data-engineering-qaqc-9TtSrW0h-py3.9/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 562, in _run_script
    exec(code, module.__dict__)
File "/app/index.py", line 56, in <module>
    run()
File "/app/index.py", line 52, in run
    app()
File "/app/src/pluto/pluto.py", line 46, in pluto
    data = get_data(branch)
File "/app/src/pluto/helpers.py", line 38, in get_data
    rv = rv | get_corrections(client, branch)
File "/app/src/pluto/helpers.py", line 51, in get_corrections
    pluto_corrections_zip = client.zip_from_DO(
File "/app/src/digital_ocean_client.py", line 93, in zip_from_DO
    buffer = BytesIO(zip_obj.get()["Body"].read())
File "/root/.cache/pypoetry/virtualenvs/data-engineering-qaqc-9TtSrW0h-py3.9/lib/python3.9/site-packages/boto3/resources/factory.py", line 580, in do_action
    response = action(self, *args, **kwargs)
File "/root/.cache/pypoetry/virtualenvs/data-engineering-qaqc-9TtSrW0h-py3.9/lib/python3.9/site-packages/boto3/resources/action.py", line 88, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
File "/root/.cache/pypoetry/virtualenvs/data-engineering-qaqc-9TtSrW0h-py3.9/lib/python3.9/site-packages/botocore/client.py", line 514, in _api_call
    return self._make_api_call(operation_name, kwargs)
File "/root/.cache/pypoetry/virtualenvs/data-engineering-qaqc-9TtSrW0h-py3.9/lib/python3.9/site-packages/botocore/client.py", line 938, in _make_api_call
    raise error_class(parsed_response, operation_name)

root causes

notes