Update upload agent version to v 1.5.33 in line with the version used for archer archiving, as per DNAnexus recommendation
Add functionality to process OncoDEEP runs (using the live OncoDEEP pan number), including oncodeep_upload v1.0.0 app
Address jinja2 dependabot security alert – update jinja2 to v3.1.4
Update naming of ‘master’ branch to ‘main’ - ‘main’ is the current widely accepted nomenclature for the primary branch
Update oncology ops email used as the recipient for the pipeline started emails
SQMove copying of the samplesheet from setoff_workflows module to demultiplexing module
Add auth string variable assignment to top of all commands files – simplifies creation / writing of commands to file
Remove TSO low throughput pan number (Pan4969)
Update duty csv version to v1.5v.0
Update samplesheet validator to v1.3.0
Fix white space addition in SQL emails (Mail clients don’t always respect <p> html tags)
Update pipeline IDs (Custom Panels – ID in use was for a previous version of the pipeline, Archer – MultiQC was not listed in previous ID, TSO – previous ID had no information listed about the app versions, OncoDEEP – new pipeline)
Set logger mode to 'w' so logging overwrites old logs
Add functionality to process development runs without UMIs (demultiplex, then upload the runfolders including BCLs, and set off fastqc / multiqc)
Add new development pan number for development runs with UMIs (Pan5227), and functionality to handle these (creates flag files to prevent demultiplexing and runfolder upload)
Add UMI dev runfolder to config
Addition to toolbox of RunfolderSamples class (derived from CollectRunfolderClass class in setoff_workflows) containing runfolder properties derived from the samplesheet, and SampleObject class originally in setoff_workflows
Alter behaviour to prevent firing of md5sum absence warnings (as there is a period of time after sequencing finishes before the file appears)
Fix so that SampleSheet is checked prior to the run finishing sequencing
Fix issue where setoff_workflows falls over for TSO runs as the samplehseet has not been copied over to the runfolder from the samplesheets directory
Fix issue where setoff workflows tries to automatically upload runs with the development pan number
Make setoff_workflows more modular: (Creation of classes per run type for collating commands generation, to make the logic easier to follow, Move dx run command functions to new build_dx_commands script, Move PipelineEmails to pipeline_emails script)
Remove Upload MultiQC as a dependency for duty_csv – addresses issue where duty_csv fails due to the cyber attack meaning that MultiQC reports cannot be uploaded to the genomics server i.e. upload multiqc fails
Sort fastqs before validation so it is easier to see how far through we are
Remove requirement to specify command line auth token for upload runfolder script – the script will only ever be run from the workstation, where the auth key file resides in the same location, so this makes it more user friendly
Improve md5sum checking behaviour – If the runfolder requires no checksum checking (i.e. is a MiSeq), or the runfolder requires checksum checking and the checksum file exists: Check sequencing is complete If the sequencer requires no checksum checking (is a MiSeq) or the integrity check passes (checksum file exists and either has been checked before and contains the checksum success message, or has not been checked before in which case the check is carried out) If the above criteria are satisfied, processing continues
Comment out addition of upload_multiqc command to dx run commands script
Update requirements file to ensure it captures all dependencies
Change missing fastqs logger message to warning
Incorporate workstation cleaner, with the addition of checking whether the runfolder is a development runfolder to prevent any deletion of dev runs
Fix LRPCR run not being identified as custom panels runtype by duty_csv, by adding the sample_prefix to the project name suffix
This update incorporates the following changes:
<p>html tags)This change is