bluesky / event-model

data model for event-based data collection and analysis
https://blueskyproject.io/event-model
BSD 3-Clause "New" or "Revised" License
15 stars 31 forks source link

Enhance ability to reach into a slice without building full data set #197

Open dylanmcreynolds opened 3 years ago

dylanmcreynolds commented 3 years ago

Attempting to view a thumbnail in a web app through databroker. I have a handler that reaches into an h5 file: handler

If I want a frame slice, and I want it lazy, I have to call something like: frame = run.stream.to_dask()['field_name']

This is nice, however...it ends up calling my handler once for every frame (in my docstream I have one event per frame). While this returns lazy dask arrays. However, there is a significant cost to calling that hanlder once for every frame. This cost isn't a huge issue for a desktop application where a couple seconds of warming up the dataset is a small price to pay for being able to randomly scroll through frames lazily. However, building a stateless web app, this unacceptable. It would be nice to have a metho that lets me dig right in to a small subset of slices to return it to the client quickly.