search

bluesky / tiled

API to structured data
https://blueskyproject.io/tiled
BSD 3-Clause "New" or "Revised" License
59 stars 50 forks source link

Better error message for when an adapter does not provide a dataframe to dask #267

Open J-avery32 opened 2 years ago

J-avery32 commented 2 years ago

Currently when I forget to turn my data into a pandas dataframe and pass it to dask I get an error message that looks like this which doesn't give much of a clue as to why there is an error. Is there a way to provide a better error message for this case?

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 43, in call_next
    message = await recv_stream.receive()
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 525, in set_cookies
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 509, in double_submit_cookie_csrf_protection
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 47, in call_next
    raise RuntimeError("No response returned.")
RuntimeError: No response returned.
danielballan commented 2 years ago

Thanks. I've been rethinking the interface to ArrayAdapter and DataFrameAdapter, with an eye to improving usability, and I'll factor this in to that effort.

danielballan commented 2 years ago

@J-avery32 Tiled v0.1.0a67 changes the __init__ signature of DataFrameAdapter and adds several more classmethod constructors. Does it improve the situation for your use case? Is the error message any better?

J-avery32 commented 2 years ago

I'll take a look at this and let you know. Probably by tomorrow or over the weekend.

J-avery32 commented 2 years ago

If I forget to return a Adapter.from_[name] function result I get this error:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 43, in call_next
    message = await recv_stream.receive()
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 525, in set_cookies
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 509, in double_submit_cookie_csrf_protection
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 462, in capture_metrics
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/compression.py", line 27, in __call__
    await responder(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/compression.py", line 48, in __call__
    await self.app(scope, receive, self.send_compressed)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/router.py", line 282, in node_metadata
    resource = construct_resource(
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/core.py", line 334, in construct_resource
    if (entry is not None) and entry.structure_family == "node":
AttributeError: 'list' object has no attribute 'structure_family'

Which is slightly better but not as explicit as would be desired.

If I forget to give the Adapter.from_[name] a pandas dataframe I get an error that looks like this:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 43, in call_next
    message = await recv_stream.receive()
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 525, in set_cookies
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 509, in double_submit_cookie_csrf_protection
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/app.py", line 462, in capture_metrics
    response = await call_next(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
    raise app_exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
    await self.app(scope, request.receive, send_stream.send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/compression.py", line 27, in __call__
    await responder(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/compression.py", line 48, in __call__
    await self.app(scope, receive, self.send_compressed)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/router.py", line 161, in node_search
    resource, metadata_stale_at, must_revalidate = construct_entries_response(
  File "/home/j/programming/work/tiled_als/venv/lib/python3.8/site-packages/tiled/server/core.py", line 110, in construct_entries_response
    if tree.structure_family != "node":
AttributeError: 'tuple' object has no attribute 'structure_family'

Which is still not great.

Ideally the error would remind me that I forgot to pass a dataframe. The first error may not be as big of a deal because it will be less common for someone to not return the Adapter.from_[name] function result. However I could totally imagine someone forgetting to convert to a dataframe before passing to the from_[name] function.