Update using-existing-devices.rst

[rosesyrett]

Related to this issue: https://github.com/bluesky/ophyd/issues/1092

It is beneficial to include in the docs some note about configuring ipython event loops so that await works as expected.

[coretl]

I was going to suggest putting this somewhere in this block: https://github.com/bluesky/ophyd/blob/5c413cc0848ef035b9839de328064d6af47b589b/docs/user_v2/examples/epics_demo.py#L13-L19

and then updating the surrounding explanation text further up the doc. I imagine quite a lot of people will copy-paste straight from the example code, so I think putting it up there makes more sense. What do you reckon?

[rosesyrett]

Yep sounds good, will change that now

[rosesyrett]

While we're at it, could I ask why you're registering RunEngine calls with < anyways? to me it reads a bit funny using that character

[rosesyrett]

Actually, I've just tried to do as you suggested here... and am getting weird errors that I think are IPython isms that I'm not understanding. If the epics_demo file looks like this:

# Import bluesky and ophyd
import matplotlib.pyplot as plt
from bluesky import RunEngine
from bluesky.callbacks.best_effort import BestEffortCallback
from bluesky.plan_stubs import mov, movr, rd  # noqa
from bluesky.plans import grid_scan  # noqa
from bluesky.run_engine import call_in_bluesky_event_loop  # noqa
from bluesky.utils import ProgressBarManager, register_transform
from IPython import get_ipython

get_ipython().run_line_magic("autoawait", "call_in_bluesky_event_loop")

from ophyd import Component, Device, EpicsSignal, EpicsSignalRO
from ophyd.v2 import epicsdemo
from ophyd.v2.core import DeviceCollector

# Create a run engine, with plotting, progressbar and transform
RE = RunEngine({}, call_returns_result=True)
bec = BestEffortCallback()
RE.subscribe(bec)
RE.waiting_hook = ProgressBarManager()
plt.ion()
register_transform("RE", prefix="<")

# Start IOC with demo pvs in subprocess
pv_prefix = epicsdemo.start_ioc_subprocess()

# Create v1 device
class OldSensor(Device):
    mode = Component(EpicsSignal, "Mode", kind="config")
    value = Component(EpicsSignalRO, "Value", kind="hinted")

det_old = OldSensor(pv_prefix, name="det_old")

# Create v2 devices
with DeviceCollector():
    det = epicsdemo.Sensor(pv_prefix)
    samp = epicsdemo.SampleStage(pv_prefix)

Then, running an ipython terminal with -i epics_demo.py and doing:

<grid_scan([det, det_old], samp.x, 1, 2, 5, samp.y, 1, 2, 5)

I get the following error:

root ➜ /workspaces/ophyd (RAYemelyanova-patch-1) $ ipython -i docs/user_v2/examples/epics_demo.py 
Python 3.9.16 (main, Feb 11 2023, 02:49:26) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.14.0 -- An enhanced Interactive Python. Type '?' for help.
Installed tk event loop hook.

In [1]: <grid_scan([det, det_old], samp.x, 1, 2, 5, samp.y, 1, 2, 5)

Transient Scan ID: 1     Time: 2023-07-20 12:51:43
Persistent Unique Scan ID: '5b72d0ab-313c-4deb-854c-3f9b8d74af2d'
New stream: 'primary'
/usr/local/lib/python3.9/site-packages/bluesky/callbacks/best_effort.py:205: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.
  fig = self._fig_factory(fig_name)
+-----------+------------+------------+------------+---------------+------------+
|   seq_num |       time |     samp-x |     samp-y | det_old_value |  det-value |
+-----------+------------+------------+------------+---------------+------------+
|         1 | 12:51:43.8 |      1.000 |      1.000 |         0.180 |      0.180 |
|         2 | 12:51:43.9 |      1.000 |      1.250 |         0.314 |      0.314 |
|         3 | 12:51:44.0 |      1.000 |      1.500 |         0.439 |      0.439 |
|         4 | 12:51:44.1 |      1.000 |      1.750 |         0.548 |      0.548 |
|         5 | 12:51:44.2 |      1.000 |      2.000 |         0.634 |      0.634 |
|         6 | 12:51:44.3 |      1.250 |      1.000 |         0.672 |      0.672 |
|         7 | 12:51:44.4 |      1.250 |      1.250 |         0.762 |      0.762 |
|         8 | 12:51:44.5 |      1.250 |      1.500 |         0.835 |      0.835 |
|         9 | 12:51:44.6 |      1.250 |      1.750 |         0.885 |      0.885 |
|        10 | 12:51:44.7 |      1.250 |      2.000 |         0.907 |      0.907 |
|        11 | 12:51:44.8 |      1.500 |      1.000 |         1.009 |      1.009 |
|        12 | 12:51:44.9 |      1.500 |      1.250 |         1.030 |      1.030 |
|        13 | 12:51:45.0 |      1.500 |      1.500 |         1.042 |      1.042 |
|        14 | 12:51:45.1 |      1.500 |      1.750 |         1.046 |      1.046 |
|        15 | 12:51:45.2 |      1.500 |      2.000 |         1.039 |      1.039 |
|        16 | 12:51:45.3 |      1.750 |      1.000 |         0.729 |      0.729 |
|        17 | 12:51:45.4 |      1.750 |      1.250 |         0.685 |      0.685 |
|        18 | 12:51:45.5 |      1.750 |      1.500 |         0.673 |      0.673 |
|        19 | 12:51:45.6 |      1.750 |      1.750 |         0.694 |      0.694 |
|        20 | 12:51:45.7 |      1.750 |      2.000 |         0.745 |      0.745 |
|        21 | 12:51:45.8 |      2.000 |      1.000 |         0.035 |      0.035 |
|        22 | 12:51:45.9 |      2.000 |      1.250 |        -0.029 |     -0.029 |
|        23 | 12:51:46.0 |      2.000 |      1.500 |         0.009 |      0.009 |
|        24 | 12:51:46.1 |      2.000 |      1.750 |         0.139 |      0.139 |
|        25 | 12:51:46.2 |      2.000 |      2.000 |         0.330 |      0.330 |
+-----------+------------+------------+------------+---------------+------------+
generator grid_scan ['5b72d0ab'] (scan num: 1)

Out[1]: RunEngineResult(run_start_uids=('5b72d0ab-313c-4deb-854c-3f9b8d74af2d',), plan_result='5b72d0ab-313c-4deb-854c-3f9b8d74af2d', exit_status='success', interrupted=False, reason='', exception=None)

In [2]: Traceback (most recent call last):
  File "/usr/local/bin/ipython", line 8, in <module>
    sys.exit(start_ipython())
  File "/usr/local/lib/python3.9/site-packages/IPython/__init__.py", line 129, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/traitlets/config/application.py", line 1043, in launch_instance
    app.start()
  File "/usr/local/lib/python3.9/site-packages/IPython/terminal/ipapp.py", line 318, in start
    self.shell.mainloop()
  File "/usr/local/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 888, in mainloop
    self.interact()
  File "/usr/local/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 873, in interact
    code = self.prompt_for_code()
  File "/usr/local/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 812, in prompt_for_code
    text = self.pt_app.prompt(
  File "/usr/local/lib/python3.9/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1035, in prompt
    return self.app.run(
  File "/usr/local/lib/python3.9/site-packages/prompt_toolkit/application/application.py", line 961, in run
    return loop.run_until_complete(coro)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1869, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/local/lib/python3.9/site-packages/prompt_toolkit/eventloop/inputhook.py", line 129, in select
    self.inputhook(InputHookContext(self._r, input_is_ready))
  File "/usr/local/lib/python3.9/site-packages/IPython/terminal/pt_inputhooks/tk.py", line 88, in inputhook
    wait_using_filehandler()
  File "/usr/local/lib/python3.9/site-packages/IPython/terminal/pt_inputhooks/tk.py", line 65, in wait_using_filehandler
    root.createfilehandler(inputhook_context.fileno(), _tkinter.READABLE, done)
RuntimeError: Calling Tcl from different apartment

If you suspect this is an IPython 8.14.0 bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/tkinter/__init__.py", line 842, in after_cancel
    data = self.tk.call('after', 'info', id)
RuntimeError: main thread is not in main loop

What do you think about this? To me it seems obvious that users should only include the 'autoawait' ipython magic if they want to play with the ophyd devices directly, not if they want to use bluesky to actually run plans with them and poke them, and plot things in the terminal. In which case I would argue, the best place for that tidbit of information is where I originally put it.

[rosesyrett]

@coretl what was the result of this discussion? Do I need to update this PR? Thanks

[coretl]

The bug is that best effort callback doesn't work with tk, only with qt. The notes say that you were going to raise a bluesky issue with this stack trace, then add pyside6 to the dev dependencies of ophyd and this PR should start passing...

[coretl]

Closing as this should be in ophyd-async but keeping the branch open so I can refer to it when solving https://github.com/bluesky/ophyd-async/issues/135