search

miso-lims / runscanner

Monitors directories on the file system containing the output from sequencing instruments (e.g. Illumina) and reports their status and metrics
https://miso-lims.github.io/runscanner/
GNU General Public License v3.0
2 stars 1 forks source link

Q|FR: Run Status for Illumina Runs #105

Closed sklages closed 4 years ago

sklages commented 4 years ago

I am using runscanner-illumina to monitor run status on the command line. That works just fine.

How does runscanner-illumina determine if a sequencer (NovaSeq, NextSeq etc) is "COMPLETED"?

The problem I see here is - at least in case of NovaSeq - that there must be another status after completing a run, that copying of the data to networkstorage has completed.

NovaSeq uses its “Universal Copy Service” to transfer the data to the output location. The CopyComplete.txt is the flag file generated at output folder to confirm data transfer to output location is completed. The same applies to NextSeq500.

That might be important when we want to run automated demultiplexing jobs after a run has been "COMPLETED". The local run status is fine, but if the data has not yet been copied completely, I cannot work with the data (on the big machines like NovaSeq)

Would that be something that could be considered by runscanner-illumina? Sure I can take care of this by myself, but it would leave runscanner-illumina somewhat "incomplete" ...

avarsava commented 4 years ago

Hi sklages,

runscanner-illumina does read illumina output, and performs some of the data retrieval for Run Scanner, however it was never intended to be a standalone tool. There is a lot of processing done after runscanner-illumina completes by the Illumina processor within Run Scanner (https://github.com/miso-lims/runscanner/blob/master/scanner/src/main/java/ca/on/oicr/gsi/runscanner/scanner/processor/DefaultIllumina.java) which is where the calculation of Run Status (COMPLETED) is done.

You can run any of the Run Scanner processors from the commandline using the ProcessRun tool, it was meant for troubleshooting however I think it will serve your purposes well: https://miso-lims.readthedocs.io/projects/runscanner/en/latest/troubleshooting/#retrieving-run-output

Please let me know if that helps or if you have any further questions about this issue!

Alexis

sklages commented 4 years ago

ok, I see. Verification of status "COMPLETED" is done later. So that's what I would also do then ..

I do get most of the infos I need from runscanner-illumina, using the RunScanner processors on the command line doesn't give me much more (that I need), it seems. I'll check it anyway.

thanks for the fast help!