search

bluesky / ophyd-async

Hardware abstraction for bluesky written using asyncio
https://blueskyproject.io/ophyd-async
BSD 3-Clause "New" or "Revised" License
12 stars 26 forks source link

Added a fixtures that checks if tasks are still running in tests #538

Closed evalott100 closed 2 months ago

evalott100 commented 3 months ago

Closes #519

evalott100 commented 3 months ago

With the additions in the above the modified test correcly failed with:

ERROR tests/core/test_status.py::test_async_status_exception_timeout - RuntimeError: Not all tasks [<Task cancelling name='Task-1481' coro=<sleep() running at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py:639> wait_for=<Future cancelled> cb=[AsyncStatusBase._run_callbacks()]>] closed during test test_async_status_exception_timeout.

Now it's been changed. We still need to track down what's causing the Task destroyed but it is pending - I would assume we need to add the same check to the RunEngine cleanup.

evalott100 commented 3 months ago

With the changes to the RunEngine we get explicit errors in the tests creating tasks that aren't awaited/cancelled.

There seems to be some nasty behavior in tests when devices connected to outside the RunEngine loop are used within the RunEngine (I've already corrected some of these tests in the same commit). I did some experimenting to see if could creatively change the RunEngine's event loop to avoid this, but no joy... I'll adjust the rest and see if I can find a way to warn the user if they use devices connected in a different event loop within the RunEngine.

https://github.com/bluesky/ophyd-async/blob/24c7ccf04eaf28370e0b57ce877b2a493209bf17/tests/conftest.py#L98-L101

=============================================== short test summary info ===============================================
ERROR tests/epics/adsimdetector/test_sim.py::test_detector_with_unnamed_or_disconnected_config_sigs[-config signal must be named before it is passed to the detector] - RuntimeError: Not all tasks {<Task pending name='Task-8496' coro=<StandardDetector.unstage() running at /scratch/twj43146/Programming/ophyd-async/src/ophyd_async/core/_detector.py:227> wait_for=<Future pending cb=[Task.task_wakeup()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/base_events.py:427> cb=[AsyncStatusBase._run_callbacks()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py:374>} closed during test test_detector_with_unnamed_or_disconnected_config_sigs[-config signal must be named before it is passed to the detector].
ERROR tests/epics/adsimdetector/test_sim.py::test_detector_with_unnamed_or_disconnected_config_sigs[some-name-config signal some-name-acquire_time must be connected before it is passed to the detector] - RuntimeError: Not all tasks {<Task pending name='Task-8596' coro=<StandardDetector.unstage() running at /scratch/twj43146/Programming/ophyd-async/src/ophyd_async/core/_detector.py:227> wait_for=<Future pending cb=[Task.task_wakeup()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/base_events.py:427> cb=[AsyncStatusBase._run_callbacks()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py:374>} closed during test test_detector_with_unnamed_or_disconnected_config_sigs[some-name-config signal some-name-acquire_time must be connected before it is passed to the detector].
ERROR tests/sim/test_sim_detector.py::test_writes_pattern_to_file - RuntimeError: Not all tasks {<Task pending name='Task-17379' coro=<PatternDetectorController._coroutine_for_image_writing() running at /scratch/twj43146/Programming/ophyd-async/src/ophyd_async/sim/demo/_pattern_detector/_pattern_detector_controller.py:55> wait_for=<Future pending cb=[Task.task_wakeup()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/base_events.py:427> cb=[AsyncStatusBase._run_callbacks()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py:374>} closed during test test_writes_pattern_to_file.
ERROR tests/sim/test_streaming_plan.py::test_streaming_plan - RuntimeError: Not all tasks {<Task pending name='Task-17410' coro=<PatternDetectorController._coroutine_for_image_writing() running at /scratch/twj43146/Programming/ophyd-async/src/ophyd_async/sim/demo/_pattern_detector/_pattern_detector_controller.py:55> wait_for=<Future pending cb=[Task.task_wakeup()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/base_events.py:427> cb=[AsyncStatusBase._run_callbacks()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py:374>} closed during test test_streaming_plan.
ERROR tests/sim/test_streaming_plan.py::test_plan - RuntimeError: Not all tasks {<Task pending name='Task-17435' coro=<PatternDetectorController._coroutine_for_image_writing() running at /scratch/twj43146/Programming/ophyd-async/src/ophyd_async/sim/demo/_pattern_detector/_pattern_detector_controller.py:55> wait_for=<Future pending cb=[Task.task_wakeup()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/base_events.py:427> cb=[AsyncStatusBase._run_callbacks()] created at /dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py:374>} closed during test test_plan.
====================================== 423 passed, 1 skipped, 5 errors in 26.88s ======================================
evalott100 commented 3 months ago

Currently, detector controller is only disarmed once:

https://github.com/bluesky/ophyd-async/blob/d7f07488fdfa976037791cbdfdb21e0f7c5f7430/src/ophyd_async/core/_detector.py#L202-L207

However it is armed in both trigger and prepare:

https://github.com/bluesky/ophyd-async/blob/d7f07488fdfa976037791cbdfdb21e0f7c5f7430/src/ophyd_async/core/_detector.py#L253-L257

https://github.com/bluesky/ophyd-async/blob/d7f07488fdfa976037791cbdfdb21e0f7c5f7430/src/ophyd_async/core/_detector.py#L299-L303

This is the cause of all the test failure's here. We should make a seperate issue for it.

evalott100 commented 2 months ago

Interestingly, the standard detector unstage error is only present if debug mode is set to true in the RE event loop, though it will still raise an error from the new clean up logic.