search

bluesky / databroker

Unified API pulling data from multiple sources
https://blueskyproject.io/databroker
BSD 3-Clause "New" or "Revised" License
35 stars 47 forks source link

Tracking issue for packaging all known handlers #533

Open danielballan opened 4 years ago

danielballan commented 4 years ago

This is a tracking issue for packaging databroker handlers. All of the handlers that were formerly developer in the databroker source tree have been moved to bluesky/area-detector-handlers. There are a couple other handlers in the wild, living mostly in profile_collection repos, that need to be captured in a proper repository, that is:

(1) in some installable package, not in profile_collection repos (2) using the new 'databroker.handlers' entrypoint so it is discoverable via databroker.core.discover_handlers().

This issue is not a blocking for releasing 1.0, but it is a blocking for deploying 1.0 at beamlines that rely on these yet-unpackaged handlers.

Known handlers

awalter-bnl commented 4 years ago

Friendly reminder, There is also a handler at IOS for their specs detector that should be included.

danielballan commented 4 years ago

Thanks. Meant to go back and find your Slack message to verify that I transcribed it right (from memory). Edited original post.

MikeHart85 commented 4 years ago

List of all the handlers I've been able to find, which are defined or referenced among the beamline profiles (searching for HandlerBase and register_handler in all profiles):

In eiger-io

In ISS profile

In TES Profile

In QAS Profile

In SRX Profile

In SIX Profile

In LIX Profile

In IOS Profile

Could not find the hxntools handlers referenced directly in any profiles, but the _handler_name / spec TPX_HDF5 is referenced in the SRX profile here, here and here... I assume that connects somehow? There may be more handlers that connect that way, that I haven't found yet.

MikeHart85 commented 4 years ago

@danielballan: How should these be packaged? All in one big package? By beamline (there seems to be some overlap/copy-pasting)? Each in separate packages?

What about the eiger-io ones? They are already a package, but it doesn't look entrypoint enabled... should it just be updated, or repackaged with the others?

danielballan commented 4 years ago

I believe that everything still relevant from eiger-io was merged into area-detector-handlers in https://github.com/bluesky/area-detector-handlers/pull/5

I would like to avoid beamline-specific packages because I think these will tend to become multi-beamline and multi-facility, even if some of them happen to be in use at one beamline today.

Everything that originates from an Area Detector (by which I mean the Area Detector IOC not "area detectors" in general sense) can go in area-detector-handlers. I believe that includes Dexela, XPRESS, Pilatus. It may include Merlin, but I'm not sure if that actually uses Area Detector; maybe @tacaswell or @mrakitin can fill in the gaps in my knowledge here.It sounds like we need a pizzabox-handlers package and a zerbabox-handlers package. Not sure where the rest of the ISS ones fit and how likely those formats are to generalize. In the short term we could go with an `iss-handlers package with the plan to split it up and/or rebrand in the future, just to keep things moving.

mrakitin commented 4 years ago

Dexela uses Mark Rivers' AreaDetector, should go there as well.

Agree about the pizzabox and zebrabox handlers.

stuartcampbell commented 4 years ago

Would it make sense to have an nsls2-detector-handlers package as a staging place for handlers that are specific to us at the moment? Or is this too much like a 'beamline package' @danielballan ?

danielballan commented 4 years ago

I like nsls2-detector-handlers as a staging place.

MikeHart85 commented 4 years ago

General update on issues encountered:

  1. ISS and QAS both have multiple PizzaBox handlers with the same name and spec, but different implementations. Not sure how to reconcile this without changing spec names or making beamline-specific repos.
  2. PilatusCBFHandler at LIX appears to operate on local and gpfs mounted files, with configurable path which is referenced in other parts of the profile as well. I'm not sure
  3. AD_EIGER_SLICE is a spec that is referenced in profiles but isn't currently provided by area-detector-handlers. This should be just a matter of adding an entrypoint associating it with EigerHandler, but need to test that it works as expected.