Fix Jasmin's failed alignment (LNA?)

[phraenquex]

Jasmin reported in Slack how her alignment failed. Here's the link: https://xchem-workspace.slack.com/archives/C069ZQUN9PY/p1720423801129119

@tdudgeon says it's for @ConorFWild

Labelling green for now.

[mwinokan]

@tdudgeon says that the LNA alignments generated include NaN coordinate values.

@ConorFWild has not yet begun investigating the work required, but it is not anticipated to cause breaking changes delaying green.

In case this is an issue with the data rather than the XCA code @ConorFWild please add informative errors.

[mwinokan]

@tdudgeon is blocked for combi-soaks (#1432) by this ticket. @ConorFWild's bandwidth is limited so @mwinokan will see if there's an alternative dataset

[mwinokan]

@ConorFWild has bandwidth this week, and will report next meeting on the a scope estimate.

@tdudgeon / Jasmin know which dataset is the problem and will report that tomorrow morning.

/dls/labxchem/data/lb32633/lb32633-6/processing/analysis/xchemalign

Slack thread

[mwinokan]

In this slack thread @ConorFWild suggests that the LNA errors are caused by inconsistent residue numbering between the reference and data to be aligned.

Jasmin will try using a different reference, and/or modify the numbering of the 6VUQ reference and re-align, next week.

[ConorFWild]

More helpful error message added in https://github.com/xchem/ligand_neighbourhood_alignment/commit/443959e91fc302b33d73524d1bab4da60e9dca83 and https://github.com/xchem/xchem-align/commit/a019e47951e26da80bca007e5a3e3163bc4489a9

[mwinokan]

From slack: Jasmin has made some changes to the numbering and is still experiencing errors.

Origin for xmap is now: [ 43.864  55.259 -10.227]
2024-08-13 07:14:09.433 | WARNING  | ligand_neighbourhood_alignment.align_xmaps:read_xmap_from_mtz:620 - Trying FWT PHWT
Origin for xmap is now: [ 43.864  55.259 -10.227]
2024-08-13 07:14:09.692 | WARNING  | ligand_neighbourhood_alignment.align_xmaps:read_xmap_from_mtz:633 - Trying DELFWT DELPHWT
Origin for xmap is now: [ 43.864  55.259 -10.227]
2024-08-13 07:14:10.010 | INFO     | ligand_neighbourhood_alignment.cli:_update:1451 - Writing to: upload_1/aligned_files/CHIKV_MacB-x0692/CHIKV_MacB-x0692_C_401_1_CHIKV_MacB-x0300+A+401+1_event.ccp4
Origin for xmap is now: [37.211 47.27  -3.184]
2024-08-13 07:14:10.277 | WARNING  | ligand_neighbourhood_alignment.align_xmaps:read_xmap_from_mtz:620 - Trying FWT PHWT
Origin for xmap is now: [37.211 47.27  -3.184]
2024-08-13 07:14:10.662 | WARNING  | ligand_neighbourhood_alignment.align_xmaps:read_xmap_from_mtz:633 - Trying DELFWT DELPHWT
Origin for xmap is now: [37.211 47.27  -3.184]
2024-08-13 07:14:11.010 | INFO     | ligand_neighbourhood_alignment.cli:_update:1451 - Writing to: upload_1/aligned_files/CHIKV_MacB-x0692/CHIKV_MacB-x0692_D_304_1_CHIKV_MacB-x0692+D+304+1_event.ccp4
Traceback (most recent call last):
  File "/dls/science/groups/i04-1/software/xchem-align/scripts/align.py", line 56, in <module>
    main()
  File "/dls/science/groups/i04-1/software/xchem-align/scripts/align.py", line 43, in main
    a.run()
  File "/dls/science/groups/i04-1/software/xchem-align/src/xchemalign/aligner.py", line 232, in run
    new_meta = self._perform_alignments(input_meta)
  File "/dls/science/groups/i04-1/software/xchem-align/src/xchemalign/aligner.py", line 465, in _perform_alignments
    updated_fs_model = _update(
  File "/dls/science/groups/i04-1/software/xchem-align/env_xchem_align/lib/python3.10/site-packages/ligand_neighbourhood_alignment/cli.py", line 1482, in _update
    __align_xmap(
  File "/dls/science/groups/i04-1/software/xchem-align/env_xchem_align/lib/python3.10/site-packages/ligand_neighbourhood_alignment/align_xmaps.py", line 582, in __align_xmap
    interpolation_range = _get_interpolation_range(neighbourhood, running_transform, reference_xmap)
  File "/dls/science/groups/i04-1/software/xchem-align/env_xchem_align/lib/python3.10/site-packages/ligand_neighbourhood_alignment/align_xmaps.py", line 238, in _get_interpolation_range
    rglb, rgub = get_grid_bounds(tlb, tub, reference_xmap)
  File "/dls/science/groups/i04-1/software/xchem-align/env_xchem_align/lib/python3.10/site-packages/ligand_neighbourhood_alignment/align_xmaps.py", line 76, in get_grid_bounds
    floor(xmap.nu * tlbf.x),
ValueError: cannot convert float NaN to integer

@ConorFWild please look at the data in /dls/labxchem/data/lb32633/lb32633-6/processing/analysis/xchemalign and let me know if you need further path/clarification

[mwinokan]

@tdudgeon will update the XCA version, and then ping Jasmin to reply.

@mwinokan can also help verify that the numbering is correct between the reference and the experimental data

[mwinokan]

@ConorFWild

Jasmin now asks:

Is it a problem that my reference structure has a H at position 0 and my datasets a M in the sequence?

TypeError: superpose_positions(): incompatible function arguments. The following argument types are supported:
    1. (pos1: List[gemmi.Position], pos2: List[gemmi.Position], weight: List[float] = []) -> gemmi.SupResult

Full log included in slack comment

[mwinokan]

@ConorFWild Jasmin's findings after using the latest XCA (still an alignment error):

https://xchem-workspace.slack.com/archives/C069ZQUN9PY/p1723554518380449?thread_ts=1723131809.831209&channel=C069ZQUN9PY&message_ts=1723554518.380449

[mwinokan]

@tdudgeon has rebuilt locally and still sees an error ("Not enough values to unpack") when testing with Ryan's data, which is a separate error as @ConorFWild has not been able to reproduce it. @tdudgeon please send @ConorFWild the error, input data and command so he can debug.

@ConorFWild has made some changes to fix the LNA issue with Jasmin's data, which have not yet been rolled out to DLS.

[mwinokan]

@tdudgeon says that @tdudgeon issue occurs when an old collator run is used, a workaround is to rerun from scratch.

@tdudgeon please rebuild the environment for DLS and notify Jasmin so she can test with her data.

[mwinokan]

From @ConorFWild in zoom chat:

What likely happened is that you had an old run that occurred before version numbers were added?

if working_fs_model.assembly_landmarks.exists():
    assembly_landmarks = ah.load_yaml(
        working_fs_model.assembly_landmarks,
        ah.dict_to_assembly_landmarks
    )

Anyway, line xchem-align:src:xchemalign:aligner.py:449 is where the issue is

This code is only invoked if you aren’t running from scratch

I.e. if working_fs_model.assembly_landmarks.exists():

@tdudgeon please see if you can find a longer term solution with a warning or fix for the aligner using old collator data

[mwinokan]

@ConorFWild is actively working on resolving this issue, but diamond's cluster is down for maintenance.

Conor says there is no likely change to the input/output format of XCA, effectively ruling out breaking changes.

[tdudgeon]

Add a simple validation to prevent aligner being run where aligner output exists: https://github.com/xchem/xchem-align/commit/8ff280289a8473c203710124da9dc5d79f05ee9f

This is not a very sophisticated check, but will suffice until we fix the problem caused by running aligner twice.

[mwinokan]

@ConorFWild ran the alignment on behalf on Jasmin with his local changes, Jasmin has successfully completed an upload to staging and has verified it looks good.

@ConorFWild will merge the changes, and @tdudgeon will update the environment

[ConorFWild]

Changes merged!

[mwinokan]

The tarball created by Jasmin this morning fails to upload:

task_status URL

{
    "started": true,
    "finished": true,
    "status": "FAILED",
    "messages": [
        "INFO: Created TargetLoader for '20240829chikv.tgz' proposal_ref='lb32633-1'",
        "INFO: Decompressing '20240829chikv.tgz'",
        "INFO: Decompressed '20240829chikv.tgz'",
        "ERROR: File meta_aligner.yaml not found in data archive",
        "ERROR: File assemblies.yaml not found in data archive",
        "Failed to process '20240829chikv.tgz'",
        "FAILED"
    ]
}

From the log files it appears the aligner did not finish running. @ConorFWild will investigate with Jasmin