Closed smondet closed 10 years ago
[sm4431@ADUAE119-LAP archive]$ nohup ./do_md5.sh 130917_SN7001341_0044_AD26YCACXX.tar.gz &
Ok
[sm4431@ADUAE119-LAP archive]$ nohup ./do_md5.sh 130917_SN7001341_0045_BD2BBLACXX.tar.gz &
Ok
In /share/archive:
03cc3c4cf77aae63ef24a29bed317ec3 130917_SN7001341_0044_AD26YCACXX.tar.gz
aad9089343fcc5ec21310b1ba3dbfbfa 130917_SN7001341_0045_BD2BBLACXX.tar.gz
$ qsub -v DIR=130917_SN7001341_0044_AD26YCACXX rsync-tar.pbs
927742.hpc0.abudhabi.nyu.edu
PBS Job Id: 927742.hpc0.abudhabi.nyu.edu
Job Name: tar-hiseq-rsync
Exec host: compute-3-5/6
Execution terminated
Exit_status=0
resources_used.cput=01:07:01
resources_used.mem=14004kb
resources_used.vmem=342300kb
resources_used.walltime=07:14:01
$ qsub -v DIR=130917_SN7001341_0045_BD2BBLACXX rsync-tar.pbs
927743.hpc0.abudhabi.nyu.edu
PBS Job Id: 927743.hpc0.abudhabi.nyu.edu
Job Name: tar-hiseq-rsync
Exec host: compute-3-10/6
Execution terminated
Exit_status=0
resources_used.cput=01:04:48
resources_used.mem=14012kb
resources_used.vmem=341408kb
resources_used.walltime=07:43:38
$ qsub -v DIR=130917_SN7001341_0044_AD26YCACXX untar.pbs
928902.hpc0.abudhabi.nyu.edu
PBS Job Id: 928902.hpc0.abudhabi.nyu.edu
Job Name: untar_$DIR
Exec host: compute-3-4/3
Execution terminated
Exit_status=0
resources_used.cput=01:22:01
resources_used.mem=4760kb
resources_used.vmem=345212kb
resources_used.walltime=08:47:03
$ qsub -v DIR=130917_SN7001341_0045_BD2BBLACXX untar.pbs
928903.hpc0.abudhabi.nyu.edu
PBS Job Id: 928903.hpc0.abudhabi.nyu.edu
Job Name: untar_$DIR
Exec host: compute-3-4/10
Execution terminated
Exit_status=0
resources_used.cput=01:29:25
resources_used.mem=4764kb
resources_used.vmem=345212kb
resources_used.walltime=08:54:29
Archival
$ qsub -v DIR=130917_SN7001341_0045_BD2BBLACXX bin/cp-to-archive.pbs
931107.hpc0.abudhabi.nyu.edu
PBS Job Id: 931107.hpc0.abudhabi.nyu.edu
Job Name: archive-tar
Exec host: compute-3-4/2
Execution terminated
Exit_status=0
resources_used.cput=00:06:52
resources_used.mem=5796kb
resources_used.vmem=330072kb
resources_used.walltime=02:09:25
$ qsub -v DIR=130917_SN7001341_0044_AD26YCACXX bin/cp-to-archive.pbs
931109.hpc0.abudhabi.nyu.edu
PBS Job Id: 931109.hpc0.abudhabi.nyu.edu
Job Name: archive-tar
Exec host: compute-3-4/3
Execution terminated
Exit_status=0
resources_used.cput=00:05:48
resources_used.mem=5800kb
resources_used.vmem=330072kb
resources_used.walltime=02:08:57
gencore@butinah-0-4:~/bin $ qsub -v DIR=130917_SN7001341_0044_AD26YCACXX tar-md5.pbs
934132.hpc0.abudhabi.nyu.edu
PBS Job Id: 934132.hpc0.abudhabi.nyu.edu
Job Name: tar-md5
Exec host: compute-1-31/0
Execution terminated
Exit_status=0
resources_used.cput=00:07:26
resources_used.mem=3568kb
resources_used.vmem=321560kb
resources_used.walltime=01:15:24
gencore@butinah-0-4:~/bin $ qsub -v DIR=130917_SN7001341_0045_BD2BBLACXX tar-md5.pbs
934133.hpc0.abudhabi.nyu.edu
PBS Job Id: 934133.hpc0.abudhabi.nyu.edu
Job Name: tar-md5
Exec host: compute-1-31/9
Execution terminated
Exit_status=0
resources_used.cput=00:07:30
resources_used.mem=3572kb
resources_used.vmem=321560kb
resources_used.walltime=01:15:41
the checksums match!
gencore@butinah-0-2:/archive/gencore/raw/hiseq-7001341 $ cat 2013-10-07.18\:1*
03cc3c4cf77aae63ef24a29bed317ec3 /archive/gencore/raw/hiseq-7001341/130917_SN7001341_0044_AD26YCACXX.tar.gz
aad9089343fcc5ec21310b1ba3dbfbfa /archive/gencore/raw/hiseq-7001341/130917_SN7001341_0045_BD2BBLACXX.tar.gz
gencore@butinah-0-2:/data/cgsb/gencore/bcl_to_fastq/130917_SN7001341_0044_AD26YCACXX $ qsub pbs/script.pbs
934614.hpc0.abudhabi.nyu.edu
gencore@butinah-0-2:/data/cgsb/gencore/bcl_to_fastq/130917_SN7001341_0045_BD2BBLACXX $ qsub pbs/script.pbs
934615.hpc0.abudhabi.nyu.edu
both failed
Dear Sebastien Mondet:
Thank you for contacting NYUAD Technology Group. We have created a request regarding High Performance Computing (Abu Dhabi) with reference number #251356. Additional details can be found below.
Description: Install bcl2fastq on BuTinah
....
Reference Number: 251356
You wrote: Hi
The genomics core has been using a new option of the sequencer that requires an additional tool for postprocessing:
It is called 'bcl2fastq' (it's different from the `configureBclToFastq.pl` that we already have as part of Casava).
See here: http://support.illumina.com/sequencing/sequencing_instruments/hiseq_2500/questions.ilmn
(look for 'bcl2fastq' in the "analysis" section)
They say the dependencies are the same as for Casava so it should be easy to install I guess.
Tell me if you need anything or have questions.
thanks
Ref:MSG1165446
→ bcl2fastq
got Installed
130917_SN7001341_0044_AD26YCACXX $ qsub pbs/script.pbs
941189.hpc0.abudhabi.nyu.edu
PBS Job Id: 941189.hpc0.abudhabi.nyu.edu
Job Name: bcl_to_fastq-44
Exec host: compute-3-14/11+compute-3-14/10+compute-3-14/9+compute-3-14/8+compute-3-14/7+compute-3-14/6+compute-3-14/5+compute-3-14/4+compute-3-14/3+compute-3-14/2+compute-3-14/1+compute-3-14/0
Execution terminated
Exit_status=0
resources_used.cput=11:01:08
resources_used.mem=400888kb
resources_used.vmem=9644988kb
resources_used.walltime=01:48:34
130917_SN7001341_0045_BD2BBLACXX $ qsub pbs/script.pbs
941190.hpc0.abudhabi.nyu.edu
PBS Job Id: 941190.hpc0.abudhabi.nyu.edu
Job Name: bcl_to_fastq-45
Exec host: compute-3-14/11+compute-3-14/10+compute-3-14/9+compute-3-14/8+compute-3-14/7+compute-3-14/6+compute-3-14/5+compute-3-14/4+compute-3-14/3+compute-3-14/2+compute-3-14/1+compute-3-14/0
Execution terminated
Exit_status=2
resources_used.cput=06:46:05
resources_used.mem=404688kb
resources_used.vmem=4748684kb
resources_used.walltime=00:58:05
---- changed the script to do only the make
gencore@butinah-0-4:/scratch/cgsb/gencore/bcl_to_fastq/130917_SN7001341_0045_BD2BBLACXX $ qsub pbs/script.pbs
942789.hpc0.abudhabi.nyu.edu
PBS Job Id: 942789.hpc0.abudhabi.nyu.edu
Job Name: bcl_to_fastq-45
Exec host: compute-1-10/11+compute-1-10/10+compute-1-10/9+compute-1-10/8+compute-1-10/7+compute-1-10/6+compute-1-10/5+compute-1-10/4+compute-1-10/3+compute-1-10/2+compute-1-10/1+compute-1-10/0
Execution terminated
Exit_status=2
resources_used.cput=00:00:28
resources_used.mem=9760kb
resources_used.vmem=328480kb
resources_used.walltime=00:05:13
Fixed the sample sheet
$ qsub pbs/script.pbs
942826.hpc0.abudhabi.nyu.edu
PBS Job Id: 942826.hpc0.abudhabi.nyu.edu
Job Name: bcl_to_fastq-45
Exec host: compute-1-10/11+compute-1-10/10+compute-1-10/9+compute-1-10/8+compute-1-10/7+compute-1-10/6+compute-1-10/5+compute-1-10/4+compute-1-10/3+compute-1-10/2+compute-1-10/1+compute-1-10/0
Execution terminated
Exit_status=0
resources_used.cput=10:48:11
resources_used.mem=480100kb
resources_used.vmem=9728712kb
resources_used.walltime=01:45:28
gencore@butinah-0-4:/scratch/cgsb/gencore/bcl_to_fastq/130917_SN7001341_0045_BD2BBLACXX $ cat SampleSheet.csv
FCID,Lane,SampleID,SampleRef,Index,Description,Control,Recipe,Operator,SampleProject
D2BBLACXX,1,Fly_H,,TAGGCATG-TAGATCGC,,N,,,Lane1
D2BBLACXX,1,Fly_J,,GGACTCCT-CTCTCTAT,,N,,,Lane1
D2BBLACXX,1,Fly_M,,AGGCAGAA-TATCCTCT,,N,,,Lane1
D2BBLACXX,2,Fly_K,,TAGGCATG-TAGATCGC,,N,,,Lane2
D2BBLACXX,2,Fly_N,,AGGCAGAA-TATCCTCT,,N,,,Lane2
D2BBLACXX,2,Fly_O,,GGACTCCT-AGAGTAGA,,N,,,Lane2
D2BBLACXX,3,Fly_P,,GGACTCCT-AGAGTAGA,,N,,,Lane3
D2BBLACXX,3,Fly_Q,,TCCTGAGC-AGAGTAGA,,N,,,Lane3
D2BBLACXX,3,Fly_R,,AGGCAGAA-TATCCTCT,,N,,,Lane3
D2BBLACXX,4,Phi_X_Control,,Undetermined,,N,,,Lane4
D2BBLACXX,5,Fly_T,,AGGCAGAA-TATCCTCT,,N,,,Lane5
D2BBLACXX,5,Fly_U,,TCCTGAGC-AGAGTAGA,,N,,,Lane5
D2BBLACXX,5,Fly_W,,GGACTCCT-AGAGTAGA,,N,,,Lane5
D2BBLACXX,6,Fly_X,,AGGCAGAA-TATCCTCT,,N,,,Lane6
D2BBLACXX,6,Fly_Y,,TCCTGAGC-AGAGTAGA,,N,,,Lane6
D2BBLACXX,6,Fly_Z,,GGACTCCT-AGAGTAGA,,N,,,Lane6
D2BBLACXX,7,Iranian_5,,TCCTGAGC-TATCCTCT,,N,,,Lane7
D2BBLACXX,7,DP1,,AGGCAGAA-AGAGTAGA,,N,,,Lane7
D2BBLACXX,8,DP14,,TAGGCATG-TAGATCGC,,N,,,Lane8
D2BBLACXX,8,DP15,,GGACTCCT-CTCTCTAT,,N,,,Lane8