search

aps-8id-dys / bluesky

XPCS bluesky instrument configuration
Other
2 stars 0 forks source link

Install environment with Python 3.10, 3.11, or 3.12 #71

Closed prjemian closed 6 days ago

prjemian commented 1 week ago

Current installation instructions say https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/README.md?plain=1#L15

Can the range of Python versions supported by this installation be expanded to 3.12? What problems appear?

prjemian commented 1 week ago

With Python 3.11 and 3.12, I have seen this error from the queueserver when opening the RunEngine environment.

Also happens with Python 3.10. It's detector-related.

prjemian commented 1 week ago

Wow! Installed apstools 1.5.3 which is quite old. Current release is 1.6.20.

prjemian commented 1 week ago

With Python 3.10, encountered this problem with queueserver:

[I 2024-09-04 16:25:18,816 bluesky_queueserver.manager.worker] Starting execution of a plan ...
[I 2024-09-04 16:25:18,816 bluesky_queueserver.manager.worker] Starting a plan 'demo_sim_1d'.
[I 2024-09-04 16:25:19,073 bluesky_queueserver.manager.plan_monitoring] New run was open: 'd7c17a2d-874e-4768-976d-cdf7dd6f6754'
Halting: skipping cleanup and marking exit_status as 'abort'...
ERROR:bluesky:Run aborted
Traceback (most recent call last):
  File "/home/beams/8IDIUSER/.conda/envs/your_env_name/lib/python3.10/site-packages/bluesky/run_engine.py", line 1507, in _run
    msg = self._plan_stack[-1].throw(stashed_exception or resp)
  File "/home/beams/8IDIUSER/.conda/envs/your_env_name/lib/python3.10/site-packages/bluesky/preprocessors.py", line 1355, in __call__
    return (yield from plan)
bluesky.utils.PlanHalt
[E 2024-09-04 16:25:19,092 bluesky_queueserver.manager.worker] The plan failed: Plan failed: Failed to import any of the following Qt binding modules: PyQt6, PySide6, PyQt5, PySide2
[I 2024-09-04 16:25:19,110 bluesky_queueserver.manager.manager] Returning current queue and running plan ...
[I 2024-09-04 16:25:19,262 bluesky_queueserver.manager.plan_monitoring] Run was closed: 'd7c17a2d-874e-4768-976d-cdf7dd6f6754'
[I 2024-09-04 16:25:20,113 bluesky_queueserver.manager.manager] Returning current queue and running plan ...
[I 2024-09-04 16:25:20,113 bluesky_queueserver.manager.manager] Returning the list of runs for the running plan ...
[I 2024-09-04 16:25:20,114 bluesky_queueserver.manager.manager] Returning plan history ...

Note that PyQt5 is installed in the environment.

(your_env_name) 8idiuser@agate ~/sandbox_bs/bluesky $ conda list qt
# packages in environment at /home/beams/8IDIUSER/.conda/envs/your_env_name:
#
# Name                    Version                   Build  Channel
pyqt                      5.15.10         py310h6a678d5_0  
pyqt5-sip                 12.13.0         py310h5eee18b_0  
qt-main                   5.15.2              h53bd1ea_10  
qtpy                      2.4.1                    pypi_0    pypi
sphinxcontrib-qthelp      2.0.0                    pypi_0    pypi
prjemian commented 1 week ago

The test plan is not calling for any PyQt5 library components. Suspect it has to do with the BestEffortCallback. Need a way for the startup to know if the queueserver is being used, so do not use a bec object.

https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/src/instrument/initialize_bs_tools.py#L35-L39

https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/src/instrument/utils/run_engine.py#L28-L29

Pass bec=None here when using queueserver. https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/src/instrument/initialize_bs_tools.py#L56

prjemian commented 1 week ago

Not there, here: https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/scripts/bs_qs_startup.py#L74-L76

prjemian commented 1 week ago

Still fails the same way with demo_sim_1d(). Try hello_world():

[I 2024-09-04 16:41:59,480 bluesky_queueserver.manager.worker] Starting a plan 'hello_world'.
[I 2024-09-04 16:41:59,620 bluesky_queueserver.manager.plan_monitoring] New run was open: '65cb38f5-3948-447d-a934-e6f5b71d2d1c'
Halting: skipping cleanup and marking exit_status as 'abort'...
ERROR:bluesky:Run aborted
Traceback (most recent call last):
  File "/home/beams/8IDIUSER/.conda/envs/your_env_name/lib/python3.10/site-packages/bluesky/run_engine.py", line 1507, in _run
    msg = self._plan_stack[-1].throw(stashed_exception or resp)
  File "/home/beams/8IDIUSER/.conda/envs/your_env_name/lib/python3.10/site-packages/bluesky/preprocessors.py", line 1355, in __call__
    return (yield from plan)
bluesky.utils.PlanHalt
[E 2024-09-04 16:41:59,621 bluesky_queueserver.manager.worker] The plan failed: Plan failed: No module named 'sip'
[I 2024-09-04 16:41:59,696 bluesky_queueserver.manager.plan_monitoring] Run was closed: '65cb38f5-3948-447d-a934-e6f5b71d2d1c'
prjemian commented 1 week ago

But sip is installed in the environment:

(your_env_name) 8idiuser@agate ~/sandbox_bs/bluesky $ conda list sip
# packages in environment at /home/beams/8IDIUSER/.conda/envs/your_env_name:
#
# Name                    Version                   Build  Channel
pyqt5-sip                 12.13.0         py310h5eee18b_0  
sip                       6.7.12          py310h6a678d5_0
prjemian commented 1 week ago

Added a couple debug prints and found this situation where two separate RE objects are created:

DEBUG: __file__='/home/beams10/8IDIUSER/sandbox_bs/bluesky/src/instrument/initialize_bs_tools.py' RE=<bluesky.run_engine.RunEngine object at 0x7f198a4caaa0>
/home/beams10/8IDIUSER/sandbox_bs/bluesky/scripts/bs_qs_startup.py
cat = <Intake catalog: 8idi_xpcs>
WARNING:aps_8id_bs_instrument.utils.run_engine:APS device not found, suspenders not installed.
DEBUG: __file__='/home/beams10/8IDIUSER/sandbox_bs/bluesky/scripts/bs_qs_startup.py' RE=<bluesky.run_engine.RunEngine object at 0x7f1a874af8e0>

That's a problem. We should have only one.

prjemian commented 1 week ago

During this import: https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/scripts/bs_qs_startup.py#L23

there are imports such as from ..initialize_bs_tools import ...: https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/src/instrument/plans/mesh_plans.py#L12 https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/src/instrument/plans/acquire_plans.py#L17 https://github.com/aps-8id-dys/bluesky/blob/af8ca959eb5032fda622f2be6d1878f70caebeda/src/instrument/plans/bdp_demo.py#L44-L45

initialize_bs_tools .py has code that builds a RE with a BestEffortCallback subscription. This bec uses PyQt5. We need a way for these plans to import the objects they need yet not create objects which we cannot use with the queueserver.

prjemian commented 1 week ago

Focus, here, on running the QS with different Python versions. So far Py3.10 is tested and working (until some failure related to #72 happens).

prjemian commented 1 week ago

Since running any plan in the QS that creates a data stream is out of the question (see #72), need a plan stub to test with the different Python versions. This is simple:

from bluesky import plan_stubs as bps

def stubtest():
    yield from bps.null()
    print(__file__)
prjemian commented 1 week ago

Successful tests with these installation instructions:

existing (Python 3.10)

conda create -y -n your_env_name "python=3.10" "pyqt=5"
conda activate your_env_name
pip install -e ".[dev]"

Python 3.11

conda create -y -n your_env_name "python=3.11" "pyqt=5"
conda activate your_env_name
pip install -e ".[dev]"

Python 3.12

conda create -y -n your_env_name "python=3.12" "pyqt=5"
conda activate your_env_name
pip install -e ".[dev]"

in sync with other bluesky packages (Python 3.12)

conda create -y -n your_env_name "python<13" "pyqt=5"
conda activate your_env_name
pip install -e ".[dev]"