Y24-026 Issues and requirements for Targeted NanoSeq

[andrewsparkes]

Description As PSD we would like to understand better the issues and requirements for the Targeted NanoSeq pipeline. Questions for users at the bottom of this description. There are also technical problems with the current design that need solutions, also see below.

Who the primary contacts are for this work Sara W Maria M

Knowledge or Stake holders Andrew S

Additional context or information Errors in production:

• on creating multiple AL Lib Dil plates from the AL Lib parent the metadata insert fails (duplicate key)
• file format error malformed utf-8, also triggered from metadata creation code

Questions for Users:

• do you 'charge and pass' on the XP plate on the initial run? to close off the library prep submission?
• do you create subsequent submissions for further runs from the AL Lib plate? If yes:
  • what plate is the submission done on?
  • what type of submission is it? (re-ISC was mentioned? it shouldn't be possible to do that on the AL Lib plate)
  • if you do re-ISC from the XP plate, how do you get from the AL Lib plate to the XP plate if you also need to upload a new customer file at the AL Lib plate?
  • do you submit only a partial subset of the samples in these repeat runs?
  • do you always take the full set of samples from the AL Lib plate through to sequencing on the initial library prep run?
  • is re-ISC functionally equivalent to repeating the initial targeted nanoseq pipeline? or does it differ in steps? (wondering if a new library prep submission at the AL Lib is equivalent)
  • currently to process a subset of the samples from the AL Lib plate you seem to upload a full customer file with dummy values for the non-required wells on the AL Lib plate, then on the Dil plate download and edit the robot driver file to remove any unneeded sample transfers, is that accurate?
    • And LIMS falsely records that you have transferred all of the occupied wells on the AL Lib plate?
    • And you get a Merged file that includes the non-required wells? Do you edit that down as well?
    • And do you then use custom pooling to filter out the non-required samples in LIMS for sequencing?

Technical issues for PSD:

• despite errors, child plates are still created and transfers set up because the errors are happening in the after_transfers method. consider options for improving this:
  • use a transaction? uses api v1 for plate creation and transfers so major redevelopment required
  • better validation up front to spot if there are any problems with the requests or proposed metadata values
• instead of inserting metadata keys, check for presence of keys and update if existing? this does not work if they want to deliberately loop back and create further runs from the AL Lib plate using the same submission requests, but it does work if there is a new submission at the AL Lib plate so we have new requests to write too. If they are doing re-ISC submissions on the XP plate though, then that is too late and the new requests would not be available on the AL Lib plate.

[andrewsparkes]

Notes from meeting 13th Mar 24

• The re-ISC route is standard for this pipeline, we can remove the non-ISC version of the Limber configuration (for PSD see config/pipelines/high_throughput_targeted_nanoseq.yml, this was likely a hangover from the duplex seq pipeline from which targeted nanoseq was originally derived).
• Subsets of samples are taken from the LTN AL Lib plate to go through several steps to the LTN Lib PCR XP plate, and then a re-ISC submission is created on that XP plate (and this partial processing is done even on the first run through).
• The customer file upload includes rows for all wells on the LTN AL Lib plate, even though they only need to progress a subset. Rows for samples that are not to be transferred are set to zero volume.
• The Hamilton driver file downloaded from Limber is edited to remove rows for wells they don't need to transfer. But LIMS tracks that all wells are transferred anyway.
• 'Charge and Pass' is clicked the first time the XP plate is reached.
• Siobhan (SSR) creates a re-ISC submission on the XP plate for the subset of samples they want to sequence.
• For subsequent iterations, they return to the AL Lib plate, upload a new version of the Customer file for the new subset of sample wells they want to process, and proceed through the library prep pipeline to the XP plate (NB. without a new library prep submission, and editing the files as before). Then the SSR creates a new re-ISC submission for that new subset of wells at the XP plate stage.
• The users think it is Ok for PSD to update/overwrite the metadata on the library prep requests for subsequent runs.
• Ideally, given the requirements above, PSD would like submissions to be made on the LTN Al Lib plate for each partial set of samples. PSD could then require the customer file to only contain rows for the relevant sample wells, could write metadata to the submission requests for only those sample wells, could transfer and bin into the LTN AL Lib Dil plate only those sample wells, provide a driver file for the Hamilton robot for only those wells, and produce a Merged file for only those wells. Reducing the workload for the lab staff and more accurately tracking the work. But probably an extra submission step for the SSR on the LTN AL Lib plate unless we could automate it.

Short term Actions:

• PSD to test the file formatting error issue and determine if there is a cause that can be fixed/avoided.
• PSD to 'fix' the current plate that is stuck (AL Lib plate SQPP-48125-S) by deleting existing metadata and re-uploading a fixed version of the file to create a working LTN AL Lib Dil plate.
• PSD to talk to Siobhan re: Submissions - how does she do this now, and could we do an extra submission to get subsets of samples from the LTN Al Lib plate through to the XP plate, then do a re-ISC submission?

Medium term Actions:

• PSD to fix the metadata insert to allow for updates in the situation where we have cancelled the previous child plate.

Longer term Actions:

• PSD to reconfigure the pipeline to more accurately reflect what the users are doing and track what is really happening with the samples.
  • Consider if we could maybe do an auto-submission at the AL Lib plate to Dil step for wells in the customer file upload they want progressed. We need a partial submission to cover the initial and subsequent runs from AL Lib to XP plate.
  • Consider the transition step to the re-ISC pipeline (suspect they fake transfer from LTN Lib PCR XP plate to LB equivalent). Is there a cleaner way to transition into ISC, or create an LTN copy?
  • Cut out Limber duplex seq configuration that isn't needed in targeted nanoseq to simplify it. e.g. always do ISC.
• PSD to improve the code in the Dil plate labware creator that identifies the request on which to store the metadata, to cope with loops back to the AL Lib plate and new submissions (latest submission requests would be in receptacle.requests_as_source not in aliquot.request, which will be the library prep request for the AL Lib plate)
• PSD to add validation to check that there is only one active request per sample well to be transferred (not zero, not 2 or more) and prevent child plate creation if not (requires a solution to create new submissions on the AL Lib Dil plate so new requests are available).
• PSD to update the integration suite test for this pipeline to match what the users are doing (subsets of samples and a loop back)

[KatyTaylor]

Write up looks good. A couple of extra things I thought of while reading through:

• They confirmed they complete a run before looping back to do a re-run - they wouldn't process these in parallel (therefore the assumption of only one active request should be sound).
• If we're considering automating the AL Lib plate submission, could consider also automating the Re-ISC submission? It could actually just be a second request type on the same AL Lib submission, as it will be the same selection of wells.

[andrewsparkes]

Notes from meeting with Siobhan AG on 18th March:

• At the start they do a library prep on all samples in the stock plate. At the AL Lib stage the customer decides what they want to sequence, and it is that subset that they are interested in through to XP and re-ISC. Sounds like it makes sense to close off the library prep at the AL Lib plate, and create a new submission for the AL Lib to XP steps.
• At the XP plate stage, Siobhan does a Sequencescape cherrypick to take selected samples from the targeted nanoseq XP plate into the LB Lib PCR-XP plate, with samples transferring across into the equivalent wells (NO rearrangement here, just cherrypicking across the subset of samples).
• The re-ISC submission is complex, with choice of projects/studies/bait libraries etc. Siobhan thinks that would be hard to automate that submission.
• There is no creation of a new submission for subsequent runs of subsets of samples between the AL Lib plate through to the XP plate (it would not make sense to make a library prep submission as libraries are already made by then). She does further cherrypicking and re-ISC submissions at the XP plate stage.
• Siobhan asks is it possible to change the config to allow re-ISC from the LTN Lib PCR XP plate, rather than have to cherrypick into the LB version and submit from there? (config of acceptable plate purposes for re-ISC submission? config of plate transfers from LTN Lib PCR XP to LB Lib PrePool? or could make LTN versions of re-ISC plate purposes?)
• An automated submission from the AL Lib plate for the part of the pipeline through to the XP plate might be practical. It would not be library prep (libraries are already made by the AL Lib plate). Ask Lab what they are doing there, we could maybe use a CustomerRequest to track this stage.
• Billing seems to be for library prep and for sequencing. Not necessarily for this intermediate section. Could ask Sara W and Rich R for further understanding of how this pipeline is charged.