Test mx_bluesky 1.0.0 as hyperion

[DominicOram]

• Running dls_dev_env.sh from deployment results in This script should be run from the 'mx-bluesky' directory - fixed in https://github.com/DiamondLightSource/mx-bluesky/pull/477

• Issue on deploy of:

  
  ERROR: while parsing the following warning configuration:
  
  ignore::sqlalchemy.exc.MovedIn20Warning

This error occurred:

Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/_pytest/config/init.py", line 1917, in parse_warning_filter category: type[Warning] = _resolve_warningcategory(category) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/_pytest/config/init.py", line 1955, in _resolve_warning_category m = import(module, None, None, [klass]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'sqlalchemy'

Fixed by ignoring this error in `pyproject.toml` - already on `main`
* Failed to build `confluent-kafka` as before. Solution is to install this package using a RHEL8 machine but everything else using i03-control - known issue that will ultimately be fixed by containerizing
* Panda issue:

ERROR:root:device coros raised unexpected exception RuntimeError Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/ophyd_async/fastcs/panda/_hdf_panda.py", line 41, in connect await fill_pvi_entries(self, self._prefix + "PVI", timeout=timeout, mock=mock) File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/ophyd_async/epics/pvi/_pvi.py", line 301, in fill_pvi_entries await _get_pvi_entries(root_entry, timeout=timeout) File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/ophyd_async/epics/pvi/_pvi.py", line 257, in _get_pvi_entries await _get_pvi_entries(sub_entry) File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/ophyd_async/epics/pvi/_pvi.py", line 260, in _get_pvi_entries _verify_common_blocks(entry, entry.common_device_type) File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/ophyd_async/epics/pvi/_pvi.py", line 102, in _verify_common_blocks raise RuntimeError( RuntimeError: sub device create_directory:<class 'typing._GenericAlias'> was not provided by pvi

Fixed by updating the panda ioc - @olliesilvester is discussing with @evalott100 and @coretl on better ways we might find out about this dependency upfront.
* Issue around `sample_shutter`:

File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/src/mx_bluesky/hyperion/device_setup_plans/utils.py", line 39, in wrapped_plan yield from plan_to_run File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/src/mx_bluesky/hyperion/experiment_plans/robot_load_then_centre_plan.py", line 269, in robot_load_then_centre_plan yield from pin_centre_then_xray_centre_plan( File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/src/mx_bluesky/hyperion/experiment_plans/pin_centre_then_xray_centre_plan.py", line 84, in pin_centre_then_xray_centre_plan yield from ispyb_activation_wrapper(_pin_centre_then_xray_centre_plan(), parameters) File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/bluesky/preprocessors.py", line 358, in run_wrapper yield from contingency_wrapper(plan, except_plan=except_plan, else_plan=close_run) File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/.venv/lib/python3.11/site-packages/bluesky/preprocessors.py", line 604, in contingency_wrapper ret = yield from plan ^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/src/mx_bluesky/hyperion/experiment_plans/pin_centre_then_xray_centre_plan.py", line 78, in _pin_centre_then_xray_centre_plan yield from detect_grid_and_do_gridscan( File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/src/mx_bluesky/hyperion/experiment_plans/grid_detect_then_xray_centre_plan.py", line 170, in detect_grid_and_do_gridscan sample_shutter=composite.sample_shutter, ^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'RobotLoadThenCentreComposite' object has no attribute 'sample_shutter'

Fixed in https://github.com/DiamondLightSource/mx-bluesky/pull/477
* We're now using properties from https://daq-config.diamond.ac.uk/ - we need to have a think about this and discuss with the scientists - https://github.com/DiamondLightSource/mx-bluesky/issues/478
* Got error of:

File "/dls_sw/i03/software/bluesky/mx_bluesky_1.0.0/hyperion/src/mx_bluesky/hyperion/external_interaction/callbacks/rotation/nexus_callback.py", line 61, in activity_gated_event assert self.writer, "Nexus writer not initialised" AssertionError: Nexus writer not initialised

This was due to missing brackets on a call to `model_dump_json`, we should have caught this in a test - Fixed in https://github.com/DiamondLightSource/mx-bluesky/pull/477
* Aperture is always going into ispyb as ROBOT_LOAD - https://ispyb.diamond.ac.uk/dc/visit/cm37235-4/id/15008409. This was due to missed early return in `_safe_move_within_datacollection_range` so `selected_aperture` not getting set. Needs tests. Also maybe we should actually read the state rather than hold it? - Fixed in https://github.com/DiamondLightSource/dodal/pull/784 follow up in https://github.com/DiamondLightSource/dodal/issues/782
* `hyperion_params` were coming through to zocalo callback as a string in the start method and failing, had to json load them. - Fixed in https://github.com/DiamondLightSource/mx-bluesky/pull/477 by putting the zocalo environment in the emitted doc so there's no need to load all the params. This matches what the XRC does.
* Issue with rotation at chi 30 https://graylog.diamond.ac.uk/messages/graylog-daq-mx-hyperion_21/252aebe1-6f88-11ef-9ed7-eaefa503891f - Needs more testing at next beam time

[coretl]

Fixed by updating the panda ioc - @olliesilvester is discussing with @evalott100 and @coretl on better ways we might find out about this dependency upfront.

Two ideas:

1. We are more explicit about the PandA IOC version requirements in the changelog
2. We annotated PandA with a custom error string containing the IOC version requirement so that it comes out in the error message.

Are either/both of those useful?

[DominicOram]

I think both would be good. For 2 are you suggesting something like the ophyd-async device reading the version from the IOC and throwing an error on connection if it's unexpected?

If we were going to do this we would have to do some clever comparison. For example, we're now using the newest IOC, which seems to work fine with the older and newer ophyd-async panda

[coretl]

I think both would be good. For 2 are you suggesting something like the ophyd-async device reading the version from the IOC and throwing an error on connection if it's unexpected?

No, something much dumber, just an error string we could put in the device that it will echo on error, so that instead of reporting "create_directory" was not provided by pvi, it would report "create_directory" was not provided by pvi: is PandABlocks-ioc at least version x.x?

[DominicOram]

My main concern is that we get issues where version X is required because of some change that would not cause an exception at connect but will cause subtle behaviour issues when it's actually used. If you think the chance of this is low then happy to just add the version string to any errors coming out of connect

[coretl]

Chance should be low of that, we haven't had any issues like that so far