Hyperion v9.3.1 testing notes

[olliesilvester]

• Panda (and hyperion) can connect after applying https://github.com/bluesky/ophyd-async/commit/1c37fd86df20fe29915b61467822306db5d27979
• Needed to remove any remaining bits of experiment_params across GDA.
• Need to add the use_gpu parameter to Hyperion
• Parameters needed to be removed from GDA following recent Hyperion changes:

beam_size_x=self.ispyb_extras.beam_size_x, beam_size_y=self.ispyb_extras.beam_size_y, focal_spot_size_x=self.ispyb_extras.focal_spot_size_x, focal_spot_size_y=self.ispyb_extras.focal_spot_size_y,

as well as resolution in ispyb_extras

• In Hyperion components.py, we had position: list[float] | NDArray = Field(default=np.array([0, 0, 0])) , and it complained when we tried to json seralise the numpy array in main, LOGGER.info(f"Started with parameters: {parameters.json(indent=2)}"). Fixed by defaulting to [0,0,0] instead

• ispyb/nexus not logging to graylog: https://github.com/DiamondLightSource/hyperion/issues/1406

• in GDA, move ispyb_experiment_type outside of ispyb_extras

Robot load

• AperturePostions need to use the one that has been instantiated properly
• mirror.apply_stripe now needs to be triggered, not set
• Looks like running on the same sample twice will put it in and out again (GDA issue?): https://github.com/DiamondLightSource/mx-bluesky/issues/240
• _sample_info in GDA isn't getting the right sampleid so subsequent rotation after robot load is failing
• Error in ispyb:

  
  [2024-05-21 15:47:43,228] Hyperion ISPyB and Zocalo callbacks plan_reactive_callback ERROR: DatabaseError(-1): Traceback (most recent call last):
  File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in _process_params
  res = [to_mysql(value) for value in res]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in <listcomp>
  res = [to_mysql(value) for value in res]
         ^^^^^^^^^^^^^^^
  File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/conversion.py", line 237, in to_mysql
  raise TypeError(
  TypeError: Python 'ca_float' cannot be converted to a MySQL type

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/ispyb/connector/mysqlsp/main.py", line 97, in call_sp_write result_args = cursor.callproc(procname=procname, args=args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 950, in callproc self.execute(f"SET {placeholders}", argvalues) File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 730, in execute psub = _ParamSubstitutor(self._process_params(params)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 571, in _process_params raise ProgrammingError( mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'ca_float' cannot be converted to a MySQL type Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in _process_params res = [to_mysql(value) for value in res] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in res = [to_mysql(value) for value in res] ^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/conversion.py", line 237, in to_mysql raise TypeError( TypeError: Python 'ca_float' cannot be converted to a MySQL type

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/ispyb/connector/mysqlsp/main.py", line 97, in call_sp_write result_args = cursor.callproc(procname=procname, args=args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 950, in callproc self.execute(f"SET {placeholders}", argvalues) File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 730, in execute psub = _ParamSubstitutor(self._process_params(params)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 571, in _process_params raise ProgrammingError( mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'ca_float' cannot be converted to a MySQL type

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/callbacks/plan_reactive_callback.py", line 50, in _run_activity_gated return self.emit(name, func(doc)) ^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/callbacks/xray_centre/ispyb_callback.py", line 138, in activity_gated_event doc = super().activity_gated_event(doc) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/callbacks/ispyb_callback_base.py", line 109, in activity_gated_event self.ispyb_ids = self.ispyb.update_deposition(self.ispyb_ids, scan_data_infos) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/ispyb/ispyb_store.py", line 77, in update_deposition return self._begin_or_update_deposition(ispyb_ids, None, scan_data_infos) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/ispyb/ispyb_store.py", line 112, in _begin_or_update_deposition new_data_collection_id, grid_id = self._store_single_scan_data( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/ispyb/ispyb_store.py", line 223, in _store_single_scan_data data_collection_id = self._store_data_collection_table( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/ispyb/ispyb_store.py", line 218, in _store_data_collection_table return self._upsert_data_collection(conn, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/contextlib.py", line 81, in inner return func(*args, **kwds) ^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/src/hyperion/external_interaction/ispyb/ispyb_store.py", line 284, in _upsert_data_collection return conn.mx_acquisition.upsert_data_collection(list(params.values())) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/ispyb/sp/acquisition.py", line 166, in upsert_data_collection return self.get_connection().call_sp_write("upsert_dc", values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/ispyb/connector/mysqlsp/main.py", line 105, in call_sp_write raise ReadWriteError( ispyb.ReadWriteError: DatabaseError(-1): Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in _process_params res = [to_mysql(value) for value in res] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in res = [to_mysql(value) for value in res] ^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/conversion.py", line 237, in to_mysql raise TypeError( TypeError: Python 'ca_float' cannot be converted to a MySQL type

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/ispyb/connector/mysqlsp/main.py", line 97, in call_sp_write result_args = cursor.callproc(procname=procname, args=args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 950, in callproc self.execute(f"SET {placeholders}", argvalues) File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 730, in execute psub = _ParamSubstitutor(self._process_params(params)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/dls_sw/i03/software/bluesky/hyperion_v9.3.1/hyperion/.venv/lib/python3.11/site-packages/mysql/connector/cursor.py", line 571, in _process_params raise ProgrammingError( mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'ca_float' cannot be converted to a MySQL type

* Robot load still not going into ispyb: https://github.com/DiamondLightSource/mx-bluesky/issues/239

[DominicOram]

Need to add the use_gpu parameter to Hyperion

Addressed in https://github.com/DiamondLightSource/hyperion/pull/1398

[DominicOram]

Also, beamsize isn't getting put in correctly https://ispyb.diamond.ac.uk/dc/visit/cm37235-2/id/14117910

[olliesilvester]

Also, beamsize isn't getting put in correctly https://ispyb.diamond.ac.uk/dc/visit/cm37235-2/id/14117910

https://github.com/DiamondLightSource/hyperion/issues/1412

[DominicOram]

Everything caught by other issues now