28-May-2013 BD27GGACXX (Ercan, Purugganan)

[agarwal]

Submission Sheet

• [x] create and email

  Transfer Data

Commands below assume, this variable has been set: RUNID=130529_SN911_0137_BD27GGACXX

• [x] TRANSFER DATA rabbot$ qsub -v DIR=$RUNID ~/bin/hiseq-rsync.pbs 13474.rabbotcluster.bio.nyu.edu -- FAILED 14320.rabbotcluster.bio.nyu.edu -- FAILED 14330.rabbotcluster.bio.nyu.edu -- FAILED 14471.rabbotcluster.bio.nyu.edu -- FAILED 14477.rabbotcluster.bio.nyu.edu -- FAILED 15154.rabbotcluster.bio.nyu.edu -- FAILED (see comments below) SUCCESS -- see comments below
• [x] BACKUP to sftp.bio rabbot:$ qsub -v DIR=$RUNID ~/bin/hiseq-backup.pbs 17358.rabbotcluster.bio.nyu.edu -- SUCCESS
• [x] COPY SAV FILES rabbot$ qsub -v DIR=$RUNID ~/bin/hiseq-rsync-SAV-files.pbs 17360.rabbotcluster.bio.nyu.edu -- FAILED (resource limit exceeded) Manually transferred all but Thumbnails.
• [x] Give Sebastien permissions: $ qsub -v DIR=$RUNID /data/cgsb/gencore-raw/bin/seb-perm.pbs 1927232.crunch.local -- SUCCESS (but need to redo due to re-transfer) 1974318.crunch.local -- SUCCESS
• [x] SECURE DATA bowery-0-3$ screen -S chattr-137 $ ~/bin/hiseq-chattr.sh $RUNID -- SUCCESS But undid this due to re-transfer. -- SUCCESS on login-0-3 $ ~/bin/hiseq-chattr.sh $RUNID -- SUCCESS

  Demultiplex

• [x] remove edit permissions on Submission Sheet
• [x] import metadata
• [x] demultiplex
• [x] fastx
• [x] review statistics

  Deliver

• [x] deliver
• [x] send emails

[agarwal]

@smondet This flowcell is ready to demultiplex.

[agarwal]

Surprisingly, although PBS job 15154 returned exit status 0, we determined that there are files that transferred incorrectly. For example, the file BaseCalls/L007/C56.1/s_7_1105.bcl has different md5sums on rabbot and bowery. We knew to investigate this particular file due to errors running CASAVA.

Changed script to use rsync's -c option, and reran:

rabbot$ qsub -v DIR=$RUNID ~/bin/hiseq-rsync.pbs 15171.rabbotcluster.bio.nyu.edu -- FAILED 15174.rabbotcluster.bio.nyu.edu -- FAILED 15179.rabbotcluster.bio.nyu.edu -- FAILED 16376.rabbotcluster.bio.nyu.edu -- FAILED 16407.rabbotcluster.bio.nyu.edu -- FAILED 17255.rabbotcluster.bio.nyu.edu -- FAILED

[agarwal]

As suggested by HPC, instead trying to run job on Bowery and pull data from rabbot:

bowery$ qsub -v DIR=$RUNID /data/cgsb/gencore-raw/bin/rsync.pbs 1969175.crunch.local -- SUCCESS

Also verified that s_7_1105.bcl has the correct md5sum now.