Closed BrunoGrandePhD closed 7 years ago
Criteria:
/hackathon/Hackathon_Project_6/DOWNLOADS/code/extractX.sh /hackathon/Hackathon_Project_6/DOWNLOADS/code/extractY.sh
These are scripts to extract X and Y from 1000G ftp site.
./extractX.sh ftp://ftp.1000genomes.ebi.ac.uk/in.bam out.bam
README about 1000 genomes GRCh38:
For High coverage PCR-free reads:
I created a file that lists the sex of each of the high coverage files that we are accessing from the ftp server.
1000genomes_GRCh38_HighCoverage_ftp_samples.txt
BAM files from two samples are being extracted now: HG00419, NA20845 4 BAM files, 1 for X, 1 for Y. in /hackathon/Hackathon_Project_6/DOWNLOADS/BAMS
48 scripts, 1 for X and 1 for Y for 24 samples in: /hackathon/Hackathon_Project_6/DOWNLOADS/code/extractAll
Make one file per sample per chrom to download 1000G BAM.
Poznik et al 2013 generate a mask for callable Y chromosome sequence.
Here is the mask file for hg19: chrY.callable.hg19.bed.txt
And I ran it through UCSC genome liftover tool to get the mask file for hg38: chrY.callable.hg38liftover.bed.txt
All 48 files (24 X chromosomes and 24 Y chromosomes) are now downloaded or in the process of downloading, and can be found in the directory:
/hackathon/Hackathon_Project_6/DOWNLOADS/BAMS
It took several attempts to get some of the files to download, but I think this is all due to memory availability. Common errors seemed to be due to limitations with the I/O memory that prevented the .cram files from being completely downloaded. The two errors when trying to extract the X and Y sequences were:
[E::hts_open_format] fail to open file 'ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/data/TSI/NA20502/high_cov_alignment/NA20502.alt_bwamem_GRCh38DH.20150917.TSI.high_coverage.cram'
And:
Aborted (core dumped) [W::bam_hdr_read] EOF marker is absent. The input is probably truncated. [E::bgzf_read] bgzf_read_block error -1 after 718072 of 760536 bytes [E::bam_hdr_read] error reading BGZF stream samtools index: "NA19017.chrX.bam" is corrupted or unsorted
http://www.internationalgenome.org/data