Fix empty sample sheet test in novaseq demux script

barrystokman commented 3 years ago

This PR adds/fixes the test for removing an empty samplesheet before fetching a new one, leading to an error email being sent without cause: ERROR starting novaseq HCTCNDSXY on clinical-preproc.scilifelab.se, line 56

How to prepare for test:

[x] Prepare test run in stage: /home/hiseq.clinical/STAGE/novaseq/runs/201203_A00689_0200_AHVKJCDRXX
[x] remove any demuxstarted.txt and SampleSheet.csv
[x] git clone master in /home/hiseq.clinical/STAGE/git/: git clone git@github.com:Clinical-Genomics/demultiplexing.git

[x] Run old bash script:

bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/

[hiseq.clinical@thalamus 201203_A00689_0200_AHVKJCDRXX]$ bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
rm: cannot remove `/home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/SampleSheet.csv': No such file or directory
cat: /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/projectlog.20201215112907.log: No such file or directory
Null message body; hope that's ok
demux sheet fetch --application nova --pad --longest HVKJCDRXX > /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/SampleSheet.csv

The defect is that the bash script is trying to remove a non-existing SampleSheet.csv, leading to an error email (rm: cannot remove/home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/SampleSheet.csv': No such file or directory`)

[x] Install new bash script by cloning this branch in /home/hiseq.clinical/STAGE/git/: git clone -b fix/demux-emails-rm-samplesheet git@github.com:Clinical-Genomics/demultiplexing.git

How to test:

TC1: no samplesheet

[x] remove any demuxstarted.txt and SampleSheet.csv
[x] Run new bash script: bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
[x] demux sheet fetch -a nova -p -L HVKJCDRXX runs
[x] no error occurs

TC2: samplesheet already exists

[x] remove demuxstarted.txt
[x] Run new bash script: bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
[x] demux sheet fetch -a nova -p -L HVKJCDRXX does NOT run
[x] no error occurs

TC3: empty samplesheet exists

[x] remove demuxstarted.txt and Samplesheet.csv, touch 'Samplesheet.csv'
[x] Run new bash script: bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
[x] demux sheet fetch -a nova -p -L HVKJCDRXX runs
[x] no error occurs

Expected test outcome: No errors occur on line 56 of the script

Review:

[x] code approved by HS
[x] tests executed by @barrystokman
[ ] "Merge and deploy" approved by Thanks for filling in who performed the code review and the test!

This version is a:

[x] PATCH - when you make backwards compatible bug fixes or documentation/instructions

barrystokman commented 3 years ago

Note:

-s : file exists AND is not empty ! -s : file does not exist OR is empty, instead of NOT (file exists AND is not empty) -> [[ -e file.txt && ! -s file.txt ]] should only return TRUE if the file exists AND is empty, which is the purpose of this check!

barrystokman commented 3 years ago

Yes, also working on the fluffy stuff but will test when I can find some time.

barrystokman commented 3 years ago

TC1 output:

[hiseq.clinical@thalamus 201203_A00689_0200_AHVKJCDRXX]$ bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
demux sheet fetch --application nova --pad --longest HVKJCDRXX > /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/SampleSheet.csv
mkdir -p /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/
date +'%Y%m%d%H%M%S' > /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/demuxstarted.txt
bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/demux-novaseq.bash /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX /home/hiseq.clinical/STAGE/novaseq/demux/ &>> /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/projectlog.20201215113443.log
rm -f /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/copycomplete.txt
date +'%Y%m%d%H%M%S' > /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/demuxcomplete.txt

:heavy_check_mark:

TC2 output:

[hiseq.clinical@thalamus 201203_A00689_0200_AHVKJCDRXX]$ bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
mkdir -p /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/
date +'%Y%m%d%H%M%S' > /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/demuxstarted.txt
bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/demux-novaseq.bash /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX /home/hiseq.clinical/STAGE/novaseq/demux/ &>> /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/projectlog.20201215113831.log
rm -f /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/copycomplete.txt
date +'%Y%m%d%H%M%S' > /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/demuxcomplete.txt

:heavy_check_mark:

TC3 output:

[hiseq.clinical@thalamus 201203_A00689_0200_AHVKJCDRXX]$ bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/checkfornewrun.bash /home/hiseq.clinical/STAGE/novaseq/runs/ /home/hiseq.clinical/STAGE/novaseq/demux/
demux sheet fetch --application nova --pad --longest HVKJCDRXX > /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/SampleSheet.csv
mkdir -p /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/
date +'%Y%m%d%H%M%S' > /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX/demuxstarted.txt
bash /home/hiseq.clinical/STAGE/git/demultiplexing/scripts/novaseq/demux-novaseq.bash /home/hiseq.clinical/STAGE/novaseq/runs//201203_A00689_0200_AHVKJCDRXX /home/hiseq.clinical/STAGE/novaseq/demux/ &>> /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/projectlog.20201215114152.log
rm -f /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/copycomplete.txt
date +'%Y%m%d%H%M%S' > /home/hiseq.clinical/STAGE/novaseq/demux//201203_A00689_0200_AHVKJCDRXX/demuxcomplete.txt

barrystokman commented 3 years ago

When testing effort >>> coding effort...

But it works and an approval would be greatly appreciated!

barrystokman commented 3 years ago

Bumped:

Deployed thalamus:

Clinical-Genomics / demultiplexing