Documentation on external assets

[danielballan]

This is a start at some much-needed documentation on how external assets work in bluesky. This is developer documentation: more practical guidance belongs in ophyd (the creation side) and databroker (the retrieval side). It is intended to explain the motivation for and details of how the relevant information is organized in documents, how the handler API works, how handler discovery works, and how the new Filler and RunRouter fit in.

-->

[danielballan]

I'm wondering about the representation of a scalar value as a vector of length 1. Do we need to make these the same, thereby losing the clear distinction between both?

@prjemian I agree that we want a clear distinction between these. I think we have it. In the event page

   # 'event_page' document
   {'data': {'random_walk:dt': [-1.0],
             'random_walk:x': [1.9221013521832928]},
   <snipped>

think of the value of 'data' as a DataFrame: a dict of lists/arrays. The fact that len(event_page['data']['random_walk:x']) == 1 tells us only that this is a length-1 Event Page. It doesn't tell us anything about the shape of the data contained therein. To get that, we might look at the number of dimensions in the first row of the page, numpy.ndim(event_page['data']['random_walk:x'][0]) which would correctly return 0. If we meant to represent a length-1 scalar there, we would have

   # 'event_page' document
   {'data': {'random_walk:dt': [[-1.0]],
             'random_walk:x': [[1.9221013521832928]]},
   <snipped>

That is, [[-1.0]] in place of [-1.0].

[danielballan]

Having addressed @prjemian comments and received two approvals, I'll merge to get this published, but comments from @dylanmcreynolds, @mrakitin, @jklynch, or anyone else are very much welcome.