bluesky / intake-bluesky

DEPRECATED: Merged into bluesky/databroker.
https://blueskyproject.io/databroker
Other
2 stars 5 forks source link

Resolve issue with intake 0.5.0+ #32

Closed danielballan closed 5 years ago

danielballan commented 5 years ago

We are currently pinned to intake 0.4.4 because intake 0.5.5 changed some behavior that causes our remote tests to fail. We will identify the problem and their either submit a patch upstream or adjust our usage, depending on the nature of the problem.

danielballan commented 5 years ago

@jklynch identified that the intake commit that breaks us is 1b8913b.

It seems like that source.describe() returns more information than it used to:

BEFORE:                                                                                                                                                                                       
{'container': 'bluesky-run-catalog', 'description': {}, 'direct_access': 'forbid', 'user_parameters': []}

AFTER:                                                                          
{'name': '3e784e06-47e6-4a57-8feb-655ba527181d', 'container': 'bluesky-run-catalog', 'plugin': "[<class 'intake_bluesky.core.RunCatalog'>]", 'description': {}, 'direct_access': 'forbid', 'us
er_parameters': [], 'metadata': {'start': {'uid': '3e784e06-47e6-4a57-8feb-655ba527181d', 'time': 1557523137.7231035, 'plan_args': {'detectors': ["SynGauss(name='det', value=1.0, timestamp=1
557523137.7085073)"], 'num': 20, 'args': ["SynAxis(prefix='', name='motor', read_attrs=['readback', 'setpoint'], configuration_attrs=['velocity', 'acceleration'])", -1, 1], 'per_step': 'None
'}, 'plan_pattern_args': {'num': 20, 'args': ["SynAxis(prefix='', name='motor', read_attrs=['readback', 'setpoint'], configuration_attrs=['velocity', 'acceleration'])", -1, 1]}, 'num_points'
: 20, 'scan_id': 1, 'plan_type': 'generator', 'plan_pattern_module': 'bluesky.plan_patterns', 'hints': {'dimensions': [[['motor'], 'primary']]}, 'plan_pattern': 'inner_product', 'detectors':
 ['det'], 'motors': ['motor'], 'plan_name': 'scan', 'num_intervals': 19}, 'stop': {'run_start': '3e784e06-47e6-4a57-8feb-655ba527181d', 'time': 1557523137.794026, 'uid': 'bed1bd97-c0ec-43a1-
b17e-88b104bc7e89', 'exit_status': 'success', 'reason': '', 'num_events': {'baseline': 2, 'primary': 20}}}, 'args': {'get_run_start': <function BlueskyJSONLCatalog._make_entries_container.<l
ocals>.Entries._doc_to_entry.<locals>.<lambda> at 0x7f35cee73ea0>, 'get_run_stop': functools.partial(<bound method BlueskyJSONLCatalog._get_run_stop of <Intake catalog: xyz>>, '3e784e06-47e6
-4a57-8feb-655ba527181d'), 'get_event_descriptors': functools.partial(<bound method BlueskyJSONLCatalog._get_event_descriptors of <Intake catalog: xyz>>, '3e784e06-47e6-4a57-8feb-655ba527181
d'), 'get_event_cursor': functools.partial(<bound method BlueskyJSONLCatalog._get_event_cursor of <Intake catalog: xyz>>, '3e784e06-47e6-4a57-8feb-655ba527181d'), 'get_event_count': functool
s.partial(<bound method BlueskyJSONLCatalog._get_event_count of <Intake catalog: xyz>>, '3e784e06-47e6-4a57-8feb-655ba527181d'), 'get_resource': functools.partial(<bound method BlueskyJSONLC
atalog._get_resource of <Intake catalog: xyz>>, '3e784e06-47e6-4a57-8feb-655ba527181d'), 'get_datum': functools.partial(<bound method BlueskyJSONLCatalog._get_datum of <Intake catalog: xyz>>
, '3e784e06-47e6-4a57-8feb-655ba527181d'), 'get_datum_cursor': functools.partial(<bound method BlueskyJSONLCatalog._get_datum_cursor of <Intake catalog: xyz>>, '3e784e06-47e6-4a57-8feb-655ba
527181d'), 'filler': <Filler>}}   

and now includes some lambdas. The server includes this output in one of its responses, and the lambda cannot be serialized.

The fix may be to stop using lambdas (probably safest anyway...). We may also want to check that intake needs to / intends to serialize all that new stuff.

danielballan commented 5 years ago

I now realize that even if we make the args serializable by pickle by removing lambda and so on, that won't fix this issue because intake needs args to be serializable by msgpack. That would require a significant API change on our side, so I think it's worth checking with intake upstream about whether this change was intentional and if it should be reverted.

danielballan commented 5 years ago

Fixed upstream. I'll leave this open as a reminder to unpin our intake dependency as soon as the next intake release is available.

danielballan commented 5 years ago

Closed by #38