NSLS-II / Bug-Reports

Unified issue-tracker for bugs in the data acquisition, management, and analysis software at NSLS-II
BSD 3-Clause "New" or "Revised" License
2 stars 5 forks source link

bluesky cannot run scans #203

Closed pelliciari closed 4 years ago

pelliciari commented 4 years ago

Hi All,

We are experiencing a serious problem with Bluesky. Suddenly we started to see this problem for no apparent reason. The count does not work presenting this error:

RE(count([sclr],num=1))
Transient Scan ID: 87558 Time: 2020-01-20 09:17:45 Persistent Unique Scan ID: 'd043056e-8fbd-4bb8-993e-0a030601d196'


DuplicateKeyError Traceback (most recent call last) ~/.ipython/profile_collection/startup/00-startup.py in ----> 1 RE(count([sclr],num=1))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in call(self, *args, **metadata_kw) 751 # it (unless it is a canceled error) 752 if exc is not None: --> 753 raise exc 754 755 if self._interrupted:

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1260 self._reason = str(err) 1261 self.log.exception("Run aborted") -> 1262 raise err 1263 finally: 1264 # Some done_callbacks may still be alive in other threads.

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1148 else: 1149 try: -> 1150 msg = self._plan_stack[-1].send(resp) 1151 # We have exhausted the top generator 1152 except StopIteration:

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in call(self, plan) 1292 plan = monitor_during_wrapper(plan, self.monitors) 1293 plan = baseline_wrapper(plan, self.baseline) -> 1294 return (yield from plan)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name) 1145 if not devices: 1146 # no-op -> 1147 return (yield from plan) 1148 else: 1149 return (yield from plan_mutator(plan, insert_baseline))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals) 788 plan1 = plan_mutator(plan, insert_after_open) 789 plan2 = plan_mutator(plan1, insert_before_close) --> 790 return (yield from plan2) 791 792

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers) 846 plan1 = plan_mutator(plan, insert_after_open) 847 plan2 = plan_mutator(plan1, insert_before_close) --> 848 return (yield from plan2) 849 850

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in call(self, plan) 1292 plan = monitor_during_wrapper(plan, self.monitors) 1293 plan = baseline_wrapper(plan, self.baseline) -> 1294 return (yield from plan)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name) 1147 return (yield from plan) 1148 else: -> 1149 return (yield from plan_mutator(plan, insert_baseline)) 1150 1151

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals) 788 plan1 = plan_mutator(plan, insert_after_open) 789 plan2 = plan_mutator(plan1, insert_before_close) --> 790 return (yield from plan2) 791 792

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers) 846 plan1 = plan_mutator(plan, insert_after_open) 847 plan2 = plan_mutator(plan1, insert_before_close) --> 848 return (yield from plan2) 849 850

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/plans.py in count(detectors, num, delay, md) 65 num=num, delay=delay)) 66 ---> 67 return (yield from inner_count()) 68 69

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, inner_kwargs) 1009 plan = gen_func(*inner_args, *inner_kwargs) 1010 plan = wrapper(plan, args, kwargs) -> 1011 return (yield from plan) 1012 return dec_inner 1013 return dec

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in stage_wrapper(plan, devices) 937 return (yield from plan) 938 --> 939 return (yield from finalize_wrapper(inner(), unstage_devices())) 940 941

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in finalize_wrapper(plan, final_plan, pause_for_debug) 498 cleanup = True 499 try: --> 500 ret = yield from plan 501 except GeneratorExit: 502 cleanup = False

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in inner() 935 def inner(): 936 yield from stage_devices() --> 937 return (yield from plan) 938 939 return (yield from finalize_wrapper(inner(), unstage_devices()))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, inner_kwargs) 1009 plan = gen_func(*inner_args, *inner_kwargs) 1010 plan = wrapper(plan, args, kwargs) -> 1011 return (yield from plan) 1012 return dec_inner 1013 return dec

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in run_wrapper(plan, md) 314 metadata to be passed into the 'open_run' message 315 """ --> 316 rs_uid = yield from open_run(md) 317 318 def except_plan(e):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/plan_stubs.py in open_run(md) 711 :func:bluesky.plans_stubs.close_run 712 """ --> 713 return (yield Msg('open_run', **(md or {}))) 714 715

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1206 # exceptions (coming in via throw) can be 1207 # raised -> 1208 new_response = yield from coro(msg) 1209 1210 # special case CancelledError and let the outer

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _open_run(self, msg) 1369 1370 doc = dict(uid=self._run_start_uid, time=ttime.time(), **md) -> 1371 yield from self.emit(DocumentNames.start, doc) 1372 self.log.debug("Emitted RunStart (uid=%r)", doc['uid']) 1373 yield from self._reset_checkpoint_state_coro()

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/asyncio/coroutines.py in coro(*args, kw) 118 @functools.wraps(func) 119 def coro(*args, *kw): --> 120 res = func(args, kw) 121 if (base_futures.isfuture(res) or inspect.isgenerator(res) or 122 isinstance(res, CoroWrapper)):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in emit(self, name, doc) 2319 "Process blocking callbacks and schedule non-blocking callbacks." 2320 schema_validators[name].validate(doc) -> 2321 self.dispatcher.process(name, doc) 2322 2323

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in process(self, name, doc) 2339 doc : dict 2340 """ -> 2341 exceptions = self.cb_registry.process(name, name.name, doc) 2342 for exc, traceback in exceptions: 2343 warn("A %r was raised during the processing of a %s "

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in process(self, sig, *args, *kwargs) 332 for cid, func in list(self.callbacks[sig].items()): 333 try: --> 334 func(args, **kwargs) 335 except ReferenceError: 336 self._remove_proxy(func)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in call(self, *args, *kwargs) 413 mtd = self.func 414 # invoke the callable and return the result --> 415 return mtd(args, **kwargs) 416 417 def eq(self, other):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/_core.py in insert(self, name, doc) 1152 return self.reg.insert_resource(ignore_duplicate_error=True, **doc) 1153 elif name in {'start', 'stop'}: -> 1154 return self.hs.insert(name, doc) 1155 else: 1156 raise ValueError

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/shim.py in insert(self, name, doc) 39 40 def insert(self, name, doc): ---> 41 return self.mds.insert(name, doc) 42 43 def find_last(self, num):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/base.py in insert(self, name, doc) 528 def insert(self, name, doc): 529 if name != 'bulk_events': --> 530 getattr(self, self._INS_METHODS[name])(**doc) 531 else: 532 for desc_uid, events in doc.items():

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/base.py in insert_run_start(self, time, uid, kwargs) 409 time=time, 410 uid=uid, --> 411 kwargs) 412 413 def insert_run_stop(self, run_start, time, uid, exit_status='success',

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/core.py in insert_run_start(run_start_col, run_start_cache, time, uid, **kwargs) 518 apply_to_dict_recursively(run_start, sanitize_np) 519 --> 520 col.insert_one(run_start) 521 522 _cache_run_start(run_start, run_start_cache)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in insert_one(self, document, bypass_document_validation, session) 691 write_concern=write_concern, 692 bypass_doc_val=bypass_document_validation, --> 693 session=session), 694 write_concern.acknowledged) 695

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert(self, docs, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session) 605 return self._insert_one( 606 docs, ordered, check_keys, manipulate, write_concern, op_id, --> 607 bypass_doc_val, session) 608 609 ids = []

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert_one(self, doc, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session) 593 594 self.__database.client._retryable_write( --> 595 acknowledged, _insert_command, session) 596 597 if not isinstance(doc, RawBSONDocument):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/mongo_client.py in _retryable_write(self, retryable, func, session) 1246 """Internal retryable write helper.""" 1247 with self._tmp_session(session) as s: -> 1248 return self._retry_with_session(retryable, func, s, None) 1249 1250 def __reset_server(self, address):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/mongo_client.py in _retry_with_session(self, retryable, func, session, bulk) 1199 raise last_error 1200 retryable = False -> 1201 return func(session, sock_info, retryable) 1202 except ServerSelectionTimeoutError: 1203 if is_retrying():

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert_command(session, sock_info, retryable_write) 590 retryable_write=retryable_write) 591 --> 592 _check_write_command_response(result) 593 594 self.__database.client._retryable_write(

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/helpers.py in _check_write_command_response(result) 215 write_errors = result.get("writeErrors") 216 if write_errors: --> 217 _raise_last_write_error(write_errors) 218 219 error = result.get("writeConcernError")

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/helpers.py in _raise_last_write_error(write_errors) 196 error = write_errors[-1] 197 if error.get("code") == 11000: --> 198 raise DuplicateKeyError(error.get("errmsg"), 11000, error) 199 raise WriteError(error.get("errmsg"), error.get("code"), error) 200

DuplicateKeyError: E11000 duplicate key error collection: metadatastore-production-v1.runstart index: uid-1 dup key: { : "d043056e-8fbd-4bb8-993e-0a030601d196" }

We already lost 12h, please advise how to fix this. This involves the sclr,rixscam and ring_curr detectors

pelliciari commented 4 years ago

Versions of DAMA software

- Bluesky    : v1.5.3
- Ophyd      : v1.3.3.post2+ga207e17
- Databroker : v0.13.0

As Maksim suggested

RE(count([sclr],num=1))
10:35:52.389868 stage -> sclr args: (), kwargs: {} 10:35:52.392553 open_run -> None args: (), kwargs: {'detectors': ['sclr'], 'num_points': 1, 'num_intervals': 0, 'plan_args': {'detectors': ["EpicsScaler(prefix='XF:02ID1-ES:1{Sclr:1}scaler1', name='sclr', read_attrs=['channels', 'channels.chan1', 'channels.chan2', 'channels.chan8', 'names', 'presets', 'gates'], configuration_attrs=['count_mode', 'delay', 'auto_count_delay', 'channels', 'names', 'names.name1', 'names.name2', 'names.name3', 'names.name4', 'names.name5', 'names.name6', 'names.name7', 'names.name8', 'names.name9', 'names.name10', 'names.name11', 'names.name12', 'names.name13', 'names.name14', 'names.name15', 'names.name16', 'names.name17', 'names.name18', 'names.name19', 'names.name20', 'names.name21', 'names.name22', 'names.name23', 'names.name24', 'names.name25', 'names.name26', 'names.name27', 'names.name28', 'names.name29', 'names.name30', 'names.name31', 'names.name32', 'time', 'freq', 'preset_time', 'auto_count_time', 'presets', 'gates', 'update_rate', 'auto_count_update_rate', 'egu'])"], 'num': 1}, 'plan_name': 'count', 'hints': {'dimensions': [(('time',), 'primary')]}} Transient Scan ID: 87560 Time: 2020-01-20 10:35:52 Persistent Unique Scan ID: '9d488bc9-09d7-448c-b9c6-497d5973ebc6' 10:35:52.647774 unstage -> sclr args: (), kwargs: {}


DuplicateKeyError Traceback (most recent call last) ~/.ipython/profile_collection/startup/00-startup.py in ----> 1 RE(count([sclr],num=1))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in call(self, *args, **metadata_kw) 751 # it (unless it is a canceled error) 752 if exc is not None: --> 753 raise exc 754 755 if self._interrupted:

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1260 self._reason = str(err) 1261 self.log.exception("Run aborted") -> 1262 raise err 1263 finally: 1264 # Some done_callbacks may still be alive in other threads.

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1148 else: 1149 try: -> 1150 msg = self._plan_stack[-1].send(resp) 1151 # We have exhausted the top generator 1152 except StopIteration:

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in call(self, plan) 1292 plan = monitor_during_wrapper(plan, self.monitors) 1293 plan = baseline_wrapper(plan, self.baseline) -> 1294 return (yield from plan)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name) 1145 if not devices: 1146 # no-op -> 1147 return (yield from plan) 1148 else: 1149 return (yield from plan_mutator(plan, insert_baseline))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals) 788 plan1 = plan_mutator(plan, insert_after_open) 789 plan2 = plan_mutator(plan1, insert_before_close) --> 790 return (yield from plan2) 791 792

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers) 846 plan1 = plan_mutator(plan, insert_after_open) 847 plan2 = plan_mutator(plan1, insert_before_close) --> 848 return (yield from plan2) 849 850

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in call(self, plan) 1292 plan = monitor_during_wrapper(plan, self.monitors) 1293 plan = baseline_wrapper(plan, self.baseline) -> 1294 return (yield from plan)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name) 1147 return (yield from plan) 1148 else: -> 1149 return (yield from plan_mutator(plan, insert_baseline)) 1150 1151

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals) 788 plan1 = plan_mutator(plan, insert_after_open) 789 plan2 = plan_mutator(plan1, insert_before_close) --> 790 return (yield from plan2) 791 792

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers) 846 plan1 = plan_mutator(plan, insert_after_open) 847 plan2 = plan_mutator(plan1, insert_before_close) --> 848 return (yield from plan2) 849 850

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/plans.py in count(detectors, num, delay, md) 65 num=num, delay=delay)) 66 ---> 67 return (yield from inner_count()) 68 69

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, inner_kwargs) 1009 plan = gen_func(*inner_args, *inner_kwargs) 1010 plan = wrapper(plan, args, kwargs) -> 1011 return (yield from plan) 1012 return dec_inner 1013 return dec

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in stage_wrapper(plan, devices) 937 return (yield from plan) 938 --> 939 return (yield from finalize_wrapper(inner(), unstage_devices())) 940 941

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in finalize_wrapper(plan, final_plan, pause_for_debug) 498 cleanup = True 499 try: --> 500 ret = yield from plan 501 except GeneratorExit: 502 cleanup = False

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in inner() 935 def inner(): 936 yield from stage_devices() --> 937 return (yield from plan) 938 939 return (yield from finalize_wrapper(inner(), unstage_devices()))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, inner_kwargs) 1009 plan = gen_func(*inner_args, *inner_kwargs) 1010 plan = wrapper(plan, args, kwargs) -> 1011 return (yield from plan) 1012 return dec_inner 1013 return dec

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in run_wrapper(plan, md) 314 metadata to be passed into the 'open_run' message 315 """ --> 316 rs_uid = yield from open_run(md) 317 318 def except_plan(e):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/plan_stubs.py in open_run(md) 711 :func:bluesky.plans_stubs.close_run 712 """ --> 713 return (yield Msg('open_run', **(md or {}))) 714 715

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1206 # exceptions (coming in via throw) can be 1207 # raised -> 1208 new_response = yield from coro(msg) 1209 1210 # special case CancelledError and let the outer

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _open_run(self, msg) 1369 1370 doc = dict(uid=self._run_start_uid, time=ttime.time(), **md) -> 1371 yield from self.emit(DocumentNames.start, doc) 1372 self.log.debug("Emitted RunStart (uid=%r)", doc['uid']) 1373 yield from self._reset_checkpoint_state_coro()

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/asyncio/coroutines.py in coro(*args, kw) 118 @functools.wraps(func) 119 def coro(*args, *kw): --> 120 res = func(args, kw) 121 if (base_futures.isfuture(res) or inspect.isgenerator(res) or 122 isinstance(res, CoroWrapper)):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in emit(self, name, doc) 2319 "Process blocking callbacks and schedule non-blocking callbacks." 2320 schema_validators[name].validate(doc) -> 2321 self.dispatcher.process(name, doc) 2322 2323

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in process(self, name, doc) 2339 doc : dict 2340 """ -> 2341 exceptions = self.cb_registry.process(name, name.name, doc) 2342 for exc, traceback in exceptions: 2343 warn("A %r was raised during the processing of a %s "

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in process(self, sig, *args, *kwargs) 332 for cid, func in list(self.callbacks[sig].items()): 333 try: --> 334 func(args, **kwargs) 335 except ReferenceError: 336 self._remove_proxy(func)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in call(self, *args, *kwargs) 413 mtd = self.func 414 # invoke the callable and return the result --> 415 return mtd(args, **kwargs) 416 417 def eq(self, other):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/_core.py in insert(self, name, doc) 1152 return self.reg.insert_resource(ignore_duplicate_error=True, **doc) 1153 elif name in {'start', 'stop'}: -> 1154 return self.hs.insert(name, doc) 1155 else: 1156 raise ValueError

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/shim.py in insert(self, name, doc) 39 40 def insert(self, name, doc): ---> 41 return self.mds.insert(name, doc) 42 43 def find_last(self, num):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/base.py in insert(self, name, doc) 528 def insert(self, name, doc): 529 if name != 'bulk_events': --> 530 getattr(self, self._INS_METHODS[name])(**doc) 531 else: 532 for desc_uid, events in doc.items():

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/base.py in insert_run_start(self, time, uid, kwargs) 409 time=time, 410 uid=uid, --> 411 kwargs) 412 413 def insert_run_stop(self, run_start, time, uid, exit_status='success',

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/core.py in insert_run_start(run_start_col, run_start_cache, time, uid, **kwargs) 518 apply_to_dict_recursively(run_start, sanitize_np) 519 --> 520 col.insert_one(run_start) 521 522 _cache_run_start(run_start, run_start_cache)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in insert_one(self, document, bypass_document_validation, session) 691 write_concern=write_concern, 692 bypass_doc_val=bypass_document_validation, --> 693 session=session), 694 write_concern.acknowledged) 695

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert(self, docs, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session) 605 return self._insert_one( 606 docs, ordered, check_keys, manipulate, write_concern, op_id, --> 607 bypass_doc_val, session) 608 609 ids = []

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert_one(self, doc, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session) 593 594 self.__database.client._retryable_write( --> 595 acknowledged, _insert_command, session) 596 597 if not isinstance(doc, RawBSONDocument):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/mongo_client.py in _retryable_write(self, retryable, func, session) 1246 """Internal retryable write helper.""" 1247 with self._tmp_session(session) as s: -> 1248 return self._retry_with_session(retryable, func, s, None) 1249 1250 def __reset_server(self, address):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/mongo_client.py in _retry_with_session(self, retryable, func, session, bulk) 1199 raise last_error 1200 retryable = False -> 1201 return func(session, sock_info, retryable) 1202 except ServerSelectionTimeoutError: 1203 if is_retrying():

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert_command(session, sock_info, retryable_write) 590 retryable_write=retryable_write) 591 --> 592 _check_write_command_response(result) 593 594 self.__database.client._retryable_write(

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/helpers.py in _check_write_command_response(result) 215 write_errors = result.get("writeErrors") 216 if write_errors: --> 217 _raise_last_write_error(write_errors) 218 219 error = result.get("writeConcernError")

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/helpers.py in _raise_last_write_error(write_errors) 196 error = write_errors[-1] 197 if error.get("code") == 11000: --> 198 raise DuplicateKeyError(error.get("errmsg"), 11000, error) 199 raise WriteError(error.get("errmsg"), error.get("code"), error) 200

DuplicateKeyError: E11000 duplicate key error collection: metadatastore-production-v1.runstart index: uid-1 dup key: { : "9d488bc9-09d7-448c-b9c6-497d5973ebc6" }

pelliciari commented 4 years ago

This is the problem with the rixscam detector

RE(count([rixscam],num=1))
10:38:21.638925 stage -> rixscam args: (), kwargs: {}

Setting "rixscam_hdf5_capture" to Done took 0.00120s (timeout=60s) Setting "rixscam_hdf5_capture" to Capturing took 0.15396s (timeout=60s) 10:38:22.160675 open_run -> None args: (), kwargs: {'detectors': ['rixscam'], 'num_points': 1, 'num_intervals': 0, 'plan_args': {'detectors': ["RIXSCam(prefix='XF:02ID1-ES{RIXSCam}:', name='rixscam', read_attrs=['xip', 'xip.count_possible_event', 'xip.count_above_threshold', 'xip.count_below_threshold', 'xip.count_neighbours', 'xip.count_event_2x2', 'xip.count_event_3x3', 'hdf5', 'hdf2'], configuration_attrs=['cam', 'cam.acquire_period', 'cam.acquire_time', 'cam.num_images', 'cam.temperature', 'cam.temperature_actual', 'cam.trigger_mode', 'centroid_enable', 'xip', 'xip.configuration_names', 'xip.port_name', 'xip.asyn_pipeline_config', 'xip.blocking_callbacks', 'xip.enable', 'xip.nd_array_port', 'xip.plugin_type', 'xip.algorithm', 'xip.output_mode', 'xip.bkgd_update_mode', 'xip.bkgd_value', 'xip.sum_3x3_threshold_min', 'xip.sum_3x3_threshold_max', 'xip.hist_start', 'xip.hist_bin_width', 'xip.hist_bin_count', 'xip.source_region', 'xip.dim0_region_start', 'xip.dim0_region_size', 'xip.dim1_region_start', 'xip.dim1_region_size', 'xip.x_expansion_factor', 'xip.y_expansion_factor', 'xip.centroid_correction', 'xip.beamline_energy', 'xip.isolinear_correction', 'xip.isolinear_coefficient_x2_1', 'xip.isolinear_coefficient_x1_1', 'xip.isolinear_coefficient_x0_1', 'xip.isolinear_coefficient_x2_2', 'xip.isolinear_coefficient_x1_2', 'xip.isolinear_coefficient_x0_2', 'xip.isolinear_threshold', 'hdf5', 'hdf5.configuration_names', 'hdf5.port_name', 'hdf5.asyn_pipeline_config', 'hdf5.blocking_callbacks', 'hdf5.enable', 'hdf5.nd_array_port', 'hdf5.plugin_type', 'hdf5.auto_increment', 'hdf5.auto_save', 'hdf5.file_format', 'hdf5.file_name', 'hdf5.file_path', 'hdf5.file_path_exists', 'hdf5.file_template', 'hdf5.file_write_mode', 'hdf5.full_file_name', 'hdf5.num_capture', 'hdf5.boundary_align', 'hdf5.boundary_threshold', 'hdf5.compression', 'hdf5.data_bits_offset', 'hdf5.extra_dim_name', 'hdf5.extra_dim_size', 'hdf5.io_speed', 'hdf5.num_col_chunks', 'hdf5.num_data_bits', 'hdf5.num_extra_dims', 'hdf5.num_frames_chunks', 'hdf5.num_frames_flush', 'hdf5.num_row_chunks', 'hdf5.run_time', 'hdf5.szip_num_pixels', 'hdf5.store_attr', 'hdf5.store_perform', 'hdf5.zlevel', 'hdf2', 'hdf2.configuration_names', 'hdf2.port_name', 'hdf2.asyn_pipeline_config', 'hdf2.blocking_callbacks', 'hdf2.enable', 'hdf2.nd_array_port', 'hdf2.plugin_type', 'hdf2.auto_increment', 'hdf2.auto_save', 'hdf2.file_format', 'hdf2.file_name', 'hdf2.file_path', 'hdf2.file_path_exists', 'hdf2.file_template', 'hdf2.file_write_mode', 'hdf2.full_file_name', 'hdf2.num_capture', 'hdf2.boundary_align', 'hdf2.boundary_threshold', 'hdf2.compression', 'hdf2.data_bits_offset', 'hdf2.extra_dim_name', 'hdf2.extra_dim_size', 'hdf2.io_speed', 'hdf2.num_col_chunks', 'hdf2.num_data_bits', 'hdf2.num_extra_dims', 'hdf2.num_frames_chunks', 'hdf2.num_frames_flush', 'hdf2.num_row_chunks', 'hdf2.run_time', 'hdf2.szip_num_pixels', 'hdf2.store_attr', 'hdf2.store_perform', 'hdf2.zlevel', 'set_node', 'sensor_region_xsize', 'sensor_region_ysize', 'sensor_region_xstart', 'sensor_region_ystart', 'sensor_binning_x', 'sensor_binning_y', 'ccd1_hv', 'ccd2_hv'])"], 'num': 1}, 'plan_name': 'count', 'hints': {'dimensions': [(('time',), 'primary')]}} Transient Scan ID: 87561 Time: 2020-01-20 10:38:22 Persistent Unique Scan ID: '79cbf7e6-0079-450b-915b-fd52e3aa8352' 10:38:22.376820 unstage -> rixscam args: (), kwargs: {} Setting "rixscam_hdf5_capture" to Done took 0.01173s (timeout=60s)


DuplicateKeyError Traceback (most recent call last) ~/.ipython/profile_collection/startup/00-startup.py in ----> 1 RE(count([rixscam],num=1))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in call(self, *args, **metadata_kw) 751 # it (unless it is a canceled error) 752 if exc is not None: --> 753 raise exc 754 755 if self._interrupted:

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1260 self._reason = str(err) 1261 self.log.exception("Run aborted") -> 1262 raise err 1263 finally: 1264 # Some done_callbacks may still be alive in other threads.

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1148 else: 1149 try: -> 1150 msg = self._plan_stack[-1].send(resp) 1151 # We have exhausted the top generator 1152 except StopIteration:

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in call(self, plan) 1292 plan = monitor_during_wrapper(plan, self.monitors) 1293 plan = baseline_wrapper(plan, self.baseline) -> 1294 return (yield from plan)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name) 1145 if not devices: 1146 # no-op -> 1147 return (yield from plan) 1148 else: 1149 return (yield from plan_mutator(plan, insert_baseline))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals) 788 plan1 = plan_mutator(plan, insert_after_open) 789 plan2 = plan_mutator(plan1, insert_before_close) --> 790 return (yield from plan2) 791 792

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers) 846 plan1 = plan_mutator(plan, insert_after_open) 847 plan2 = plan_mutator(plan1, insert_before_close) --> 848 return (yield from plan2) 849 850

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in call(self, plan) 1292 plan = monitor_during_wrapper(plan, self.monitors) 1293 plan = baseline_wrapper(plan, self.baseline) -> 1294 return (yield from plan)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in baseline_wrapper(plan, devices, name) 1147 return (yield from plan) 1148 else: -> 1149 return (yield from plan_mutator(plan, insert_baseline)) 1150 1151

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in monitor_during_wrapper(plan, signals) 788 plan1 = plan_mutator(plan, insert_after_open) 789 plan2 = plan_mutator(plan1, insert_before_close) --> 790 return (yield from plan2) 791 792

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in fly_during_wrapper(plan, flyers) 846 plan1 = plan_mutator(plan, insert_after_open) 847 plan2 = plan_mutator(plan1, insert_before_close) --> 848 return (yield from plan2) 849 850

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 167 continue 168 else: --> 169 raise ex 170 # if inserting / mutating, put new generator on the stack 171 # and replace the current msg with the first element from the

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 120 ret = result_stack.pop() 121 try: --> 122 msg = plan_stack[-1].send(ret) 123 except StopIteration as e: 124 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/plans.py in count(detectors, num, delay, md) 65 num=num, delay=delay)) 66 ---> 67 return (yield from inner_count()) 68 69

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, inner_kwargs) 1009 plan = gen_func(*inner_args, *inner_kwargs) 1010 plan = wrapper(plan, args, kwargs) -> 1011 return (yield from plan) 1012 return dec_inner 1013 return dec

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in stage_wrapper(plan, devices) 937 return (yield from plan) 938 --> 939 return (yield from finalize_wrapper(inner(), unstage_devices())) 940 941

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in finalize_wrapper(plan, final_plan, pause_for_debug) 498 cleanup = True 499 try: --> 500 ret = yield from plan 501 except GeneratorExit: 502 cleanup = False

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in inner() 935 def inner(): 936 yield from stage_devices() --> 937 return (yield from plan) 938 939 return (yield from finalize_wrapper(inner(), unstage_devices()))

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in dec_inner(*inner_args, inner_kwargs) 1009 plan = gen_func(*inner_args, *inner_kwargs) 1010 plan = wrapper(plan, args, kwargs) -> 1011 return (yield from plan) 1012 return dec_inner 1013 return dec

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in run_wrapper(plan, md) 314 metadata to be passed into the 'open_run' message 315 """ --> 316 rs_uid = yield from open_run(md) 317 318 def except_plan(e):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/plan_stubs.py in open_run(md) 711 :func:bluesky.plans_stubs.close_run 712 """ --> 713 return (yield Msg('open_run', **(md or {}))) 714 715

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 75 # if we have a stashed exception, pass it along 76 try: ---> 77 msg = plan_stack[-1].throw(exception) 78 except StopIteration as e: 79 # discard the exhausted generator

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in single_gen(msg) 134 the input message 135 ''' --> 136 return (yield msg) 137 138

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/preprocessors.py in plan_mutator(plan, msg_proc) 192 try: 193 # yield out the 'current message' and collect the return --> 194 inner_ret = yield msg 195 except GeneratorExit: 196 # special case GeneratorExit. We must clean up all of our plans

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _run(self) 1206 # exceptions (coming in via throw) can be 1207 # raised -> 1208 new_response = yield from coro(msg) 1209 1210 # special case CancelledError and let the outer

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in _open_run(self, msg) 1369 1370 doc = dict(uid=self._run_start_uid, time=ttime.time(), **md) -> 1371 yield from self.emit(DocumentNames.start, doc) 1372 self.log.debug("Emitted RunStart (uid=%r)", doc['uid']) 1373 yield from self._reset_checkpoint_state_coro()

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/asyncio/coroutines.py in coro(*args, kw) 118 @functools.wraps(func) 119 def coro(*args, *kw): --> 120 res = func(args, kw) 121 if (base_futures.isfuture(res) or inspect.isgenerator(res) or 122 isinstance(res, CoroWrapper)):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in emit(self, name, doc) 2319 "Process blocking callbacks and schedule non-blocking callbacks." 2320 schema_validators[name].validate(doc) -> 2321 self.dispatcher.process(name, doc) 2322 2323

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/run_engine.py in process(self, name, doc) 2339 doc : dict 2340 """ -> 2341 exceptions = self.cb_registry.process(name, name.name, doc) 2342 for exc, traceback in exceptions: 2343 warn("A %r was raised during the processing of a %s "

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in process(self, sig, *args, *kwargs) 332 for cid, func in list(self.callbacks[sig].items()): 333 try: --> 334 func(args, **kwargs) 335 except ReferenceError: 336 self._remove_proxy(func)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/bluesky/utils.py in call(self, *args, *kwargs) 413 mtd = self.func 414 # invoke the callable and return the result --> 415 return mtd(args, **kwargs) 416 417 def eq(self, other):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/_core.py in insert(self, name, doc) 1152 return self.reg.insert_resource(ignore_duplicate_error=True, **doc) 1153 elif name in {'start', 'stop'}: -> 1154 return self.hs.insert(name, doc) 1155 else: 1156 raise ValueError

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/shim.py in insert(self, name, doc) 39 40 def insert(self, name, doc): ---> 41 return self.mds.insert(name, doc) 42 43 def find_last(self, num):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/base.py in insert(self, name, doc) 528 def insert(self, name, doc): 529 if name != 'bulk_events': --> 530 getattr(self, self._INS_METHODS[name])(**doc) 531 else: 532 for desc_uid, events in doc.items():

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/base.py in insert_run_start(self, time, uid, kwargs) 409 time=time, 410 uid=uid, --> 411 kwargs) 412 413 def insert_run_stop(self, run_start, time, uid, exit_status='success',

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/databroker/headersource/core.py in insert_run_start(run_start_col, run_start_cache, time, uid, **kwargs) 518 apply_to_dict_recursively(run_start, sanitize_np) 519 --> 520 col.insert_one(run_start) 521 522 _cache_run_start(run_start, run_start_cache)

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in insert_one(self, document, bypass_document_validation, session) 691 write_concern=write_concern, 692 bypass_doc_val=bypass_document_validation, --> 693 session=session), 694 write_concern.acknowledged) 695

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert(self, docs, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session) 605 return self._insert_one( 606 docs, ordered, check_keys, manipulate, write_concern, op_id, --> 607 bypass_doc_val, session) 608 609 ids = []

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert_one(self, doc, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session) 593 594 self.__database.client._retryable_write( --> 595 acknowledged, _insert_command, session) 596 597 if not isinstance(doc, RawBSONDocument):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/mongo_client.py in _retryable_write(self, retryable, func, session) 1246 """Internal retryable write helper.""" 1247 with self._tmp_session(session) as s: -> 1248 return self._retry_with_session(retryable, func, s, None) 1249 1250 def __reset_server(self, address):

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/mongo_client.py in _retry_with_session(self, retryable, func, session, bulk) 1199 raise last_error 1200 retryable = False -> 1201 return func(session, sock_info, retryable) 1202 except ServerSelectionTimeoutError: 1203 if is_retrying():

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/collection.py in _insert_command(session, sock_info, retryable_write) 590 retryable_write=retryable_write) 591 --> 592 _check_write_command_response(result) 593 594 self.__database.client._retryable_write(

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/helpers.py in _check_write_command_response(result) 215 write_errors = result.get("writeErrors") 216 if write_errors: --> 217 _raise_last_write_error(write_errors) 218 219 error = result.get("writeConcernError")

/opt/conda_envs/collection-2019-3.0-six/lib/python3.7/site-packages/pymongo/helpers.py in _raise_last_write_error(write_errors) 196 error = write_errors[-1] 197 if error.get("code") == 11000: --> 198 raise DuplicateKeyError(error.get("errmsg"), 11000, error) 199 raise WriteError(error.get("errmsg"), error.get("code"), error) 200

DuplicateKeyError: E11000 duplicate key error collection: metadatastore-production-v1.runstart index: uid-1 dup key: { : "79cbf7e6-0079-450b-915b-fd52e3aa8352" }

mrakitin commented 4 years ago

Sorry, pressed the wrong button. The problem is resolved over the phone call. Johnny @pelliciari will update with the details of the issues shortly.

mrakitin commented 4 years ago

There were 2 issues caused by the explicit %run:

Please close the corresponding Trac ticket. I am closing this issue as resolved.

pelliciari commented 4 years ago

We spotted the problems. One was an error in the definition of a motor. The second problem was due to an unnecessary running of the startup script creating a conflict into the data broker