livedata should work with bluesky scans

prjemian commented 5 years ago

Now that USAXS is preparing to use bluesky, make sure the livedata can access bluesky's databroker for data content. Also, the SPEC data files from bluesky should have sufficient content for livedata.

prjemian commented 5 years ago

There is little metadata in the SPEC data files generated from bluesky. Mostly, the SPEC data file records the primary scan data. At least each scan's uid is provided in a comment near the start of each scan:

#S 1145  myPlan()
#D Mon Jan 28 12:48:09 2019
#C Mon Jan 28 12:48:09 2019.  plan_type = generator
#C Mon Jan 28 12:48:09 2019.  uid = ef98648a-8e3a-4e7e-ac99-3290c9b5fca7
#C Mon Jan 28 12:48:09 2019.  this is a start document comment

The uid is the index to use with databroker to access other content (including any metadata) for the scan.

prjemian commented 5 years ago

prjemian commented 5 years ago

With prjemian/spec2nexus#132 and additions to the metadata such as

https://github.com/APS-USAXS/ipython-usaxs/blob/775a01394a59a8cfeb482d460d8a314cebddc511/profile_bluesky/startup/50-plans.py#L545-L547

we now have SPEC scan data with useful metadata that are identified as key-value pairs (a Python dict):

#S 1  tune_mr()
#D Thu May 02 10:51:07 2019
#C Thu May 02 10:51:07 2019.  plan_type = generator
#C Thu May 02 10:51:07 2019.  uid = 4d6ba919-f8dd-4d26-bd4f-955b0ecafe97
#MD uid = 4d6ba919-f8dd-4d26-bd4f-955b0ecafe97
#MD APSTOOLS_VERSION = 1.1.0
#MD BLUESKY_VERSION = 1.5.2
#MD EPICS_CA_MAX_ARRAY_BYTES = 1280000
#MD EPICS_HOST_ARCH = linux-x86_64
#MD OPHYD_VERSION = 1.3.3
#MD beamline_id = APS USAXS 9-ID-C
#MD datetime = 2019-05-02 10:51:07.906554
#MD login_id = usaxs@usaxscontrol.xray.aps.anl.gov
#MD pid = 13765
#MD proposal_id = testing Bluesky installation
#MD purpose = tuner
#MD tune_md = {'width': -0.004, 'initial_position': 8.824977, 'time_iso8601': '2019-05-02 10:51:07.918077'}
#MD tune_parameters = {'num': 31, 'width': -0.004, 'initial_position': 8.824977, 'peak_choice': 'com', 'x_axis': 'm_stage_r', 'y_axis': 'I0_USAXS'}
#N 31
#L Epoch_float  Epoch  m_stage_r  m_stage_r_user_setpoint  m_stage_r_soft_limit_lo  m_stage_r_soft_limit_hi  seconds  I0_USAXS  I00_USAXS  PD_USAXS  TR diode  I000  scaler0_time  scaler0_display_rate
1.6743626594543457 2 8.826977 8.826977000000001 -7.856115 37.143885 1000000.0 127.0 3142.0 575.0 1.0 0.0 0.1 5.0
2.0949885845184326 2 8.826844 8.826843666666667 -7.856115 37.143885 1000000.0 127.0 3142.0 568.0 1.0 0.0 0.1 5.0
2.5119998455047607 3 8.826711 8.826710333333335 -7.856115 37.143885 1000000.0 127.0 3143.0 487.0 1.0 0.0 0.1 5.0
2.9395253658294678 3 8.826577 8.826577 -7.856115 37.143885 1000000.0 127.0 3142.0 509.0 1.0 0.0 0.1 5.0
3.3453152179718018 3 8.826443999999999 8.826443666666668 -7.856115 37.143885 1000000.0 127.0 3142.0 481.0 1.0 0.0 0.1 5.0
3.7644267082214355 4 8.826311 8.826310333333334 -7.856115 37.143885 1000000.0 127.0 3143.0 473.0 1.0 0.0 0.1 5.0
4.1785054206848145 4 8.826177 8.826177000000001 -7.856115 37.143885 1000000.0 127.0 3143.0 510.0 1.0 0.0 0.1 5.0
4.595493316650391 5 8.826044 8.826043666666667 -7.856115 37.143885 1000000.0 127.0 3143.0 463.0 1.0 0.0 0.1 5.0
4.995158910751343 5 8.825911 8.825910333333335 -7.856115 37.143885 1000000.0 127.0 3143.0 469.0 1.0 0.0 0.1 5.0
5.4104228019714355 5 8.825776999999999 8.825777 -7.856115 37.143885 1000000.0 125.0 3140.0 22284.0 1.0 0.0 0.1 5.0
5.827091932296753 6 8.825644 8.825643666666668 -7.856115 37.143885 1000000.0 127.0 3142.0 603.0 1.0 0.0 0.1 5.0
6.245729684829712 6 8.825510999999999 8.825510333333334 -7.856115 37.143885 1000000.0 126.0 3142.0 25.0 1.0 0.0 0.1 5.0
6.64563775062561 7 8.825377 8.825377000000001 -7.856115 37.143885 1000000.0 127.0 3143.0 464.0 1.0 0.0 0.1 5.0
7.060603380203247 7 8.825244 8.825243666666667 -7.856115 37.143885 1000000.0 127.0 3143.0 998.0 1.0 0.0 0.1 5.0
7.477765798568726 7 8.825111 8.825110333333335 -7.856115 37.143885 1000000.0 126.0 3143.0 2266.0 1.0 0.0 0.1 5.0
7.895208120346069 8 8.824977 8.824977 -7.856115 37.143885 1000000.0 127.0 3143.0 543.0 1.0 0.0 0.1 5.0
8.294854164123535 8 8.824843999999999 8.824843666666666 -7.856115 37.143885 1000000.0 127.0 3143.0 467.0 1.0 0.0 0.1 5.0
8.712139129638672 9 8.824711 8.824710333333334 -7.856115 37.143885 1000000.0 127.0 3142.0 483.0 1.0 0.0 0.1 5.0
9.128334283828735 9 8.824577 8.824577 -7.856115 37.143885 1000000.0 127.0 3142.0 498.0 1.0 0.0 0.1 5.0
9.545223951339722 10 8.824444 8.824443666666667 -7.856115 37.143885 1000000.0 127.0 3143.0 473.0 1.0 0.0 0.1 5.0
9.945016860961914 10 8.824311 8.824310333333333 -7.856115 37.143885 1000000.0 127.0 3142.0 464.0 1.0 0.0 0.1 5.0
10.363540887832642 10 8.824176999999999 8.824177 -7.856115 37.143885 1000000.0 126.0 3140.0 22257.0 1.0 0.0 0.1 5.0
10.778327226638794 11 8.824044 8.824043666666666 -7.856115 37.143885 1000000.0 126.0 3143.0 558.0 1.0 0.0 0.1 5.0
11.194773197174072 11 8.823910999999999 8.823910333333334 -7.856115 37.143885 1000000.0 127.0 3142.0 471.0 1.0 0.0 0.1 5.0
11.596132278442383 12 8.823777 8.823777 -7.856115 37.143885 1000000.0 127.0 3143.0 461.0 1.0 0.0 0.1 5.0
12.012276887893677 12 8.823644 8.823643666666667 -7.856115 37.143885 1000000.0 127.0 3143.0 465.0 1.0 0.0 0.1 5.0
12.445284366607666 12 8.823511 8.823510333333333 -7.856115 37.143885 1000000.0 127.0 3143.0 486.0 1.0 0.0 0.1 5.0
12.860543966293335 13 8.823377 8.823377 -7.856115 37.143885 1000000.0 127.0 3142.0 970.0 1.0 0.0 0.1 5.0
13.281290531158447 13 8.823243999999999 8.823243666666666 -7.856115 37.143885 1000000.0 126.0 3142.0 2485.0 1.0 0.0 0.1 5.0
13.69581389427185 14 8.823111 8.823110333333334 -7.856115 37.143885 1000000.0 127.0 3142.0 602.0 1.0 0.0 0.1 5.0
14.110772132873535 14 8.822977 8.822977 -7.856115 37.143885 1000000.0 126.0 3143.0 2412.0 1.0 0.0 0.1 5.0
#C Thu May 02 10:51:22 2019.  num_events_baseline = 2
#C Thu May 02 10:51:22 2019.  num_events_primary = 31
#C Thu May 02 10:51:22 2019.  exit_status = success

prjemian commented 5 years ago

more work to do on the bluesky plans but this looks finished here

APS-USAXS / livedata

livedata should work with bluesky scans #9