search

NorwegianVeterinaryInstitute / DemultiplexRawSequenceData

A workflow automation script: demultiplex the library sequence, run quality checks, deliver to archiving and processing afterwards
GNU General Public License v3.0
1 stars 0 forks source link

feature request: Detect failed runs for MiSeq/NexSeq #37

Open georgemarselis-nvi opened 1 year ago

georgemarselis-nvi commented 1 year ago

No idea how, I have to ask Cathrine, but here is an idea

First file the directory + 35 hours: if no RTAComplete.txt has shown up then it is a failed run. Email person responsible for failed runs.

Points to improve:

  1. See if the 35 hour window can be minimized
arvindsundaram commented 1 year ago
  1. Calculate the total number of reads from 1-4 in RunParameters.xml file.
  2. Follow the read in realtime by checking the filename/foldername produced under Data/Intensities/Basecalls/L001 folder. Do remember that the first 26 files/folders will appear few hours after run has started.
  3. Once the number of cycles are complete (calcuated from 1. and found in 2.), then look for RTAComplete.txt in the RunFolder.
  4. Wait for CopyComplete.txt to appear before proceeding with Demultiplexing (this is fairly new).
  5. During point 2, new files/folders should appear every 30 mins or so. If nothing new appears for 2 hours, the run is probably failed. This applies only after cycle 25.
karinlag commented 1 year ago

Great info, @arvindsundaram ! Question: the CopyComplete file, does that indicate that a run has been completely transferred from the sequencer to the server?

Also, just to get things stupidly clear, in which order ar ethe various xml and txt files produced (the ones indicating the progress of a run, that is).

arvindsundaram commented 1 year ago

@karinlag Yes, you are correct. One can start demultiplexing when "RTAComplete.txt" file appears but need to wait to move/copy the run until "CopyComplete.txt" file appears. The second appears approx. 20 mins after the first one in MiSeq.

I do not know the order but timestamps might not tell the story as these files might be re-written. RunParameters.xml is probably written at the start of the run.