Set up AWS megatests - Githubissues

ewels commented 3 years ago

AWS megatests is now running nicely and we’re trying to set up all (most) nf-core pipelines to run a big dataset. We need to identify a set of public data to run benchmarks for the pipeline.

The idea is that this will run automatically for every release of the nf-core/methylseq pipeline. The results will then be publicly accessible from s3 and viewable through the website: https://nf-co.re/methylseq/results - this means that people can manually compare differences in output between pipeline releases if they wish.

We need a dataset that is as “normal” as possible, mouse or human, sequenced relatively recently and with a bunch of replicates etc. It can be a fairly large project

I'm hoping that @FelixKrueger can help here, but suggestions from anyone and everyone are more than welcome! ✋🏻

In practical terms, once decided we need to:

[ ] Upload the FastQ files to s3: s3://nf-core-awsmegatests/methylseq/input_data/ (I can help with this)
[ ] Update test_full.config to work with these file paths
[ ] Check .github/workflows/awsfulltest.yml (should be no changes required I think?)
[ ] Merge, and try running the dev branch manually

FelixKrueger commented 3 years ago

I have found a dataset that potentially ticks several of these boxes:

it human data
it is fairly large
there is bulk as well as subclone data (biological replicates?)
it is fairly recent
it is normal (directional) BS-seq

Here is the link to GEO: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE120794

Here is a list of the files involved:

download.sh

```bash #!/usr/bin/env bash curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961084/SRR7961084_1.fastq.gz -o SRR7961084_GSM3415648_Clone_E7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961084/SRR7961084_2.fastq.gz -o SRR7961084_GSM3415648_Clone_E7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961085/SRR7961085_1.fastq.gz -o SRR7961085_GSM3415649_Clone_F1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961085/SRR7961085_2.fastq.gz -o SRR7961085_GSM3415649_Clone_F1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961086/SRR7961086_1.fastq.gz -o SRR7961086_GSM3415650_Clone_F4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961086/SRR7961086_2.fastq.gz -o SRR7961086_GSM3415650_Clone_F4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961087/SRR7961087_1.fastq.gz -o SRR7961087_GSM3415651_Clone_G2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961087/SRR7961087_2.fastq.gz -o SRR7961087_GSM3415651_Clone_G2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961082/SRR7961082_1.fastq.gz -o SRR7961082_GSM3415646_Clone_B8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961082/SRR7961082_2.fastq.gz -o SRR7961082_GSM3415646_Clone_B8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961083/SRR7961083_1.fastq.gz -o SRR7961083_GSM3415647_Clone_E4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961083/SRR7961083_2.fastq.gz -o SRR7961083_GSM3415647_Clone_E4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961090/SRR7961090_1.fastq.gz -o SRR7961090_GSM3415654_MShef11_low_oxygen_N10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961090/SRR7961090_2.fastq.gz -o SRR7961090_GSM3415654_MShef11_low_oxygen_N10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961092/SRR7961092_1.fastq.gz -o SRR7961092_GSM3415656_MShef11_low_oxygen_N2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961092/SRR7961092_2.fastq.gz -o SRR7961092_GSM3415656_MShef11_low_oxygen_N2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961088/SRR7961088_1.fastq.gz -o SRR7961088_GSM3415652_Clone_G8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961088/SRR7961088_2.fastq.gz -o SRR7961088_GSM3415652_Clone_G8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961089/SRR7961089_1.fastq.gz -o SRR7961089_GSM3415653_MShef11_bulk_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961089/SRR7961089_2.fastq.gz -o SRR7961089_GSM3415653_MShef11_bulk_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961091/SRR7961091_1.fastq.gz -o SRR7961091_GSM3415655_MShef11_low_oxygen_N1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961091/SRR7961091_2.fastq.gz -o SRR7961091_GSM3415655_MShef11_low_oxygen_N1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961093/SRR7961093_1.fastq.gz -o SRR7961093_GSM3415657_MShef11_low_oxygen_N3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961093/SRR7961093_2.fastq.gz -o SRR7961093_GSM3415657_MShef11_low_oxygen_N3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961095/SRR7961095_1.fastq.gz -o SRR7961095_GSM3415659_MShef11_low_oxygen_N5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961095/SRR7961095_2.fastq.gz -o SRR7961095_GSM3415659_MShef11_low_oxygen_N5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961097/SRR7961097_1.fastq.gz -o SRR7961097_GSM3415661_MShef11_low_oxygen_N7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961097/SRR7961097_2.fastq.gz -o SRR7961097_GSM3415661_MShef11_low_oxygen_N7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961096/SRR7961096_1.fastq.gz -o SRR7961096_GSM3415660_MShef11_low_oxygen_N6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961096/SRR7961096_2.fastq.gz -o SRR7961096_GSM3415660_MShef11_low_oxygen_N6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961094/SRR7961094_1.fastq.gz -o SRR7961094_GSM3415658_MShef11_low_oxygen_N4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961094/SRR7961094_2.fastq.gz -o SRR7961094_GSM3415658_MShef11_low_oxygen_N4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961098/SRR7961098_1.fastq.gz -o SRR7961098_GSM3415662_MShef11_low_oxygen_N8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961098/SRR7961098_2.fastq.gz -o SRR7961098_GSM3415662_MShef11_low_oxygen_N8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961099/SRR7961099_1.fastq.gz -o SRR7961099_GSM3415663_MShef11_low_oxygen_Q10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961099/SRR7961099_2.fastq.gz -o SRR7961099_GSM3415663_MShef11_low_oxygen_Q10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961100/SRR7961100_1.fastq.gz -o SRR7961100_GSM3415664_MShef11_low_oxygen_Q11_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961100/SRR7961100_2.fastq.gz -o SRR7961100_GSM3415664_MShef11_low_oxygen_Q11_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961103/SRR7961103_1.fastq.gz -o SRR7961103_GSM3415667_MShef11_low_oxygen_Q2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961103/SRR7961103_2.fastq.gz -o SRR7961103_GSM3415667_MShef11_low_oxygen_Q2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961101/SRR7961101_1.fastq.gz -o SRR7961101_GSM3415665_MShef11_low_oxygen_Q12_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961101/SRR7961101_2.fastq.gz -o SRR7961101_GSM3415665_MShef11_low_oxygen_Q12_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961102/SRR7961102_1.fastq.gz -o SRR7961102_GSM3415666_MShef11_low_oxygen_Q1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961102/SRR7961102_2.fastq.gz -o SRR7961102_GSM3415666_MShef11_low_oxygen_Q1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961105/SRR7961105_1.fastq.gz -o SRR7961105_GSM3415669_MShef11_low_oxygen_Q4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961105/SRR7961105_2.fastq.gz -o SRR7961105_GSM3415669_MShef11_low_oxygen_Q4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961104/SRR7961104_1.fastq.gz -o SRR7961104_GSM3415668_MShef11_low_oxygen_Q3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961104/SRR7961104_2.fastq.gz -o SRR7961104_GSM3415668_MShef11_low_oxygen_Q3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961106/SRR7961106_1.fastq.gz -o SRR7961106_GSM3415670_MShef11_low_oxygen_Q5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961106/SRR7961106_2.fastq.gz -o SRR7961106_GSM3415670_MShef11_low_oxygen_Q5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961107/SRR7961107_1.fastq.gz -o SRR7961107_GSM3415671_MShef11_low_oxygen_Q6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961107/SRR7961107_2.fastq.gz -o SRR7961107_GSM3415671_MShef11_low_oxygen_Q6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961108/SRR7961108_1.fastq.gz -o SRR7961108_GSM3415672_MShef11_low_oxygen_Q7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961108/SRR7961108_2.fastq.gz -o SRR7961108_GSM3415672_MShef11_low_oxygen_Q7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961109/SRR7961109_1.fastq.gz -o SRR7961109_GSM3415673_MShef11_low_oxygen_Q8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961109/SRR7961109_2.fastq.gz -o SRR7961109_GSM3415673_MShef11_low_oxygen_Q8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961110/SRR7961110_1.fastq.gz -o SRR7961110_GSM3415674_MShef11_low_oxygen_Q9_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961110/SRR7961110_2.fastq.gz -o SRR7961110_GSM3415674_MShef11_low_oxygen_Q9_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961113/SRR7961113_1.fastq.gz -o SRR7961113_GSM3415677_MShef11_standard_O2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961113/SRR7961113_2.fastq.gz -o SRR7961113_GSM3415677_MShef11_standard_O2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961112/SRR7961112_1.fastq.gz -o SRR7961112_GSM3415676_MShef11_standard_O1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961112/SRR7961112_2.fastq.gz -o SRR7961112_GSM3415676_MShef11_standard_O1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961111/SRR7961111_1.fastq.gz -o SRR7961111_GSM3415675_MShef11_standard_O10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961111/SRR7961111_2.fastq.gz -o SRR7961111_GSM3415675_MShef11_standard_O10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961114/SRR7961114_1.fastq.gz -o SRR7961114_GSM3415678_MShef11_standard_O3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961114/SRR7961114_2.fastq.gz -o SRR7961114_GSM3415678_MShef11_standard_O3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961115/SRR7961115_1.fastq.gz -o SRR7961115_GSM3415679_MShef11_standard_O4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961115/SRR7961115_2.fastq.gz -o SRR7961115_GSM3415679_MShef11_standard_O4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961116/SRR7961116_1.fastq.gz -o SRR7961116_GSM3415680_MShef11_standard_O5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961116/SRR7961116_2.fastq.gz -o SRR7961116_GSM3415680_MShef11_standard_O5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961117/SRR7961117_1.fastq.gz -o SRR7961117_GSM3415681_MShef11_standard_O6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961117/SRR7961117_2.fastq.gz -o SRR7961117_GSM3415681_MShef11_standard_O6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961118/SRR7961118_1.fastq.gz -o SRR7961118_GSM3415682_MShef11_standard_O7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961118/SRR7961118_2.fastq.gz -o SRR7961118_GSM3415682_MShef11_standard_O7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961119/SRR7961119_1.fastq.gz -o SRR7961119_GSM3415683_MShef11_standard_O8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961119/SRR7961119_2.fastq.gz -o SRR7961119_GSM3415683_MShef11_standard_O8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961120/SRR7961120_1.fastq.gz -o SRR7961120_GSM3415684_MShef11_standard_O9_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961120/SRR7961120_2.fastq.gz -o SRR7961120_GSM3415684_MShef11_standard_O9_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961121/SRR7961121_1.fastq.gz -o SRR7961121_GSM3415685_MShef11_standard_P10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961121/SRR7961121_2.fastq.gz -o SRR7961121_GSM3415685_MShef11_standard_P10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961122/SRR7961122_1.fastq.gz -o SRR7961122_GSM3415686_MShef11_standard_P1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961122/SRR7961122_2.fastq.gz -o SRR7961122_GSM3415686_MShef11_standard_P1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961123/SRR7961123_1.fastq.gz -o SRR7961123_GSM3415687_MShef11_standard_P2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961123/SRR7961123_2.fastq.gz -o SRR7961123_GSM3415687_MShef11_standard_P2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961124/SRR7961124_1.fastq.gz -o SRR7961124_GSM3415688_MShef11_standard_P3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961124/SRR7961124_2.fastq.gz -o SRR7961124_GSM3415688_MShef11_standard_P3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961126/SRR7961126_1.fastq.gz -o SRR7961126_GSM3415690_MShef11_standard_P5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961126/SRR7961126_2.fastq.gz -o SRR7961126_GSM3415690_MShef11_standard_P5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961125/SRR7961125_1.fastq.gz -o SRR7961125_GSM3415689_MShef11_standard_P4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961125/SRR7961125_2.fastq.gz -o SRR7961125_GSM3415689_MShef11_standard_P4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961127/SRR7961127_1.fastq.gz -o SRR7961127_GSM3415691_MShef11_standard_P6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961127/SRR7961127_2.fastq.gz -o SRR7961127_GSM3415691_MShef11_standard_P6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961128/SRR7961128_1.fastq.gz -o SRR7961128_GSM3415692_MShef11_standard_P8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961128/SRR7961128_2.fastq.gz -o SRR7961128_GSM3415692_MShef11_standard_P8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961129/SRR7961129_1.fastq.gz -o SRR7961129_GSM3415693_MShef11_standard_P9_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961129/SRR7961129_2.fastq.gz -o SRR7961129_GSM3415693_MShef11_standard_P9_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961130/SRR7961130_1.fastq.gz -o SRR7961130_GSM3415694_MShef11_Y27632_K10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961130/SRR7961130_2.fastq.gz -o SRR7961130_GSM3415694_MShef11_Y27632_K10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961132/SRR7961132_1.fastq.gz -o SRR7961132_GSM3415696_MShef11_Y27632_K2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961132/SRR7961132_2.fastq.gz -o SRR7961132_GSM3415696_MShef11_Y27632_K2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961133/SRR7961133_1.fastq.gz -o SRR7961133_GSM3415697_MShef11_Y27632_K3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961133/SRR7961133_2.fastq.gz -o SRR7961133_GSM3415697_MShef11_Y27632_K3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961131/SRR7961131_1.fastq.gz -o SRR7961131_GSM3415695_MShef11_Y27632_K1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961131/SRR7961131_2.fastq.gz -o SRR7961131_GSM3415695_MShef11_Y27632_K1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961134/SRR7961134_1.fastq.gz -o SRR7961134_GSM3415698_MShef11_Y27632_K4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961134/SRR7961134_2.fastq.gz -o SRR7961134_GSM3415698_MShef11_Y27632_K4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961136/SRR7961136_1.fastq.gz -o SRR7961136_GSM3415700_MShef11_Y27632_K6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961136/SRR7961136_2.fastq.gz -o SRR7961136_GSM3415700_MShef11_Y27632_K6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961135/SRR7961135_1.fastq.gz -o SRR7961135_GSM3415699_MShef11_Y27632_K5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961135/SRR7961135_2.fastq.gz -o SRR7961135_GSM3415699_MShef11_Y27632_K5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961138/SRR7961138_1.fastq.gz -o SRR7961138_GSM3415702_MShef11_Y27632_K8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961138/SRR7961138_2.fastq.gz -o SRR7961138_GSM3415702_MShef11_Y27632_K8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961137/SRR7961137_1.fastq.gz -o SRR7961137_GSM3415701_MShef11_Y27632_K7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961137/SRR7961137_2.fastq.gz -o SRR7961137_GSM3415701_MShef11_Y27632_K7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961140/SRR7961140_1.fastq.gz -o SRR7961140_GSM3415704_MShef11_Y27632_L10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961140/SRR7961140_2.fastq.gz -o SRR7961140_GSM3415704_MShef11_Y27632_L10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961139/SRR7961139_1.fastq.gz -o SRR7961139_GSM3415703_MShef11_Y27632_K9_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961139/SRR7961139_2.fastq.gz -o SRR7961139_GSM3415703_MShef11_Y27632_K9_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961141/SRR7961141_1.fastq.gz -o SRR7961141_GSM3415705_MShef11_Y27632_L1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961141/SRR7961141_2.fastq.gz -o SRR7961141_GSM3415705_MShef11_Y27632_L1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961142/SRR7961142_1.fastq.gz -o SRR7961142_GSM3415706_MShef11_Y27632_L2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961142/SRR7961142_2.fastq.gz -o SRR7961142_GSM3415706_MShef11_Y27632_L2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961143/SRR7961143_1.fastq.gz -o SRR7961143_GSM3415707_MShef11_Y27632_L3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961143/SRR7961143_2.fastq.gz -o SRR7961143_GSM3415707_MShef11_Y27632_L3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961144/SRR7961144_1.fastq.gz -o SRR7961144_GSM3415708_MShef11_Y27632_L4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961144/SRR7961144_2.fastq.gz -o SRR7961144_GSM3415708_MShef11_Y27632_L4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961146/SRR7961146_1.fastq.gz -o SRR7961146_GSM3415710_MShef11_Y27632_L6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961146/SRR7961146_2.fastq.gz -o SRR7961146_GSM3415710_MShef11_Y27632_L6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961145/SRR7961145_1.fastq.gz -o SRR7961145_GSM3415709_MShef11_Y27632_L5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961145/SRR7961145_2.fastq.gz -o SRR7961145_GSM3415709_MShef11_Y27632_L5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961147/SRR7961147_1.fastq.gz -o SRR7961147_GSM3415711_MShef11_Y27632_L7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961147/SRR7961147_2.fastq.gz -o SRR7961147_GSM3415711_MShef11_Y27632_L7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961149/SRR7961149_1.fastq.gz -o SRR7961149_GSM3415713_MShef11_Y27632_L9_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961149/SRR7961149_2.fastq.gz -o SRR7961149_GSM3415713_MShef11_Y27632_L9_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961148/SRR7961148_1.fastq.gz -o SRR7961148_GSM3415712_MShef11_Y27632_L8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961148/SRR7961148_2.fastq.gz -o SRR7961148_GSM3415712_MShef11_Y27632_L8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961150/SRR7961150_1.fastq.gz -o SRR7961150_GSM3415714_MShef4_bulk_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961150/SRR7961150_2.fastq.gz -o SRR7961150_GSM3415714_MShef4_bulk_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961151/SRR7961151_1.fastq.gz -o SRR7961151_GSM3415715_MShef4_J10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961151/SRR7961151_2.fastq.gz -o SRR7961151_GSM3415715_MShef4_J10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961152/SRR7961152_1.fastq.gz -o SRR7961152_GSM3415716_MShef4_J11_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961152/SRR7961152_2.fastq.gz -o SRR7961152_GSM3415716_MShef4_J11_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961154/SRR7961154_1.fastq.gz -o SRR7961154_GSM3415718_MShef4_J13_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961154/SRR7961154_2.fastq.gz -o SRR7961154_GSM3415718_MShef4_J13_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961153/SRR7961153_1.fastq.gz -o SRR7961153_GSM3415717_MShef4_J12_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961153/SRR7961153_2.fastq.gz -o SRR7961153_GSM3415717_MShef4_J12_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961155/SRR7961155_1.fastq.gz -o SRR7961155_GSM3415719_MShef4_J14_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961155/SRR7961155_2.fastq.gz -o SRR7961155_GSM3415719_MShef4_J14_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961156/SRR7961156_1.fastq.gz -o SRR7961156_GSM3415720_MShef4_J15_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961156/SRR7961156_2.fastq.gz -o SRR7961156_GSM3415720_MShef4_J15_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961157/SRR7961157_1.fastq.gz -o SRR7961157_GSM3415721_MShef4_J16_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961157/SRR7961157_2.fastq.gz -o SRR7961157_GSM3415721_MShef4_J16_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961158/SRR7961158_1.fastq.gz -o SRR7961158_GSM3415722_MShef4_J17_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961158/SRR7961158_2.fastq.gz -o SRR7961158_GSM3415722_MShef4_J17_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961159/SRR7961159_1.fastq.gz -o SRR7961159_GSM3415723_MShef4_J18_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961159/SRR7961159_2.fastq.gz -o SRR7961159_GSM3415723_MShef4_J18_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961161/SRR7961161_1.fastq.gz -o SRR7961161_GSM3415725_MShef4_J1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961161/SRR7961161_2.fastq.gz -o SRR7961161_GSM3415725_MShef4_J1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961160/SRR7961160_1.fastq.gz -o SRR7961160_GSM3415724_MShef4_J19_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961160/SRR7961160_2.fastq.gz -o SRR7961160_GSM3415724_MShef4_J19_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961162/SRR7961162_1.fastq.gz -o SRR7961162_GSM3415726_MShef4_J20_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961162/SRR7961162_2.fastq.gz -o SRR7961162_GSM3415726_MShef4_J20_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961164/SRR7961164_1.fastq.gz -o SRR7961164_GSM3415728_MShef4_J3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961164/SRR7961164_2.fastq.gz -o SRR7961164_GSM3415728_MShef4_J3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961163/SRR7961163_1.fastq.gz -o SRR7961163_GSM3415727_MShef4_J2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961163/SRR7961163_2.fastq.gz -o SRR7961163_GSM3415727_MShef4_J2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961167/SRR7961167_1.fastq.gz -o SRR7961167_GSM3415731_MShef4_J6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961167/SRR7961167_2.fastq.gz -o SRR7961167_GSM3415731_MShef4_J6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961165/SRR7961165_1.fastq.gz -o SRR7961165_GSM3415729_MShef4_J4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961165/SRR7961165_2.fastq.gz -o SRR7961165_GSM3415729_MShef4_J4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961166/SRR7961166_1.fastq.gz -o SRR7961166_GSM3415730_MShef4_J5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961166/SRR7961166_2.fastq.gz -o SRR7961166_GSM3415730_MShef4_J5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961168/SRR7961168_1.fastq.gz -o SRR7961168_GSM3415732_MShef4_J7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961168/SRR7961168_2.fastq.gz -o SRR7961168_GSM3415732_MShef4_J7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961170/SRR7961170_1.fastq.gz -o SRR7961170_GSM3415734_MShef4_J9_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961170/SRR7961170_2.fastq.gz -o SRR7961170_GSM3415734_MShef4_J9_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961169/SRR7961169_1.fastq.gz -o SRR7961169_GSM3415733_MShef4_J8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961169/SRR7961169_2.fastq.gz -o SRR7961169_GSM3415733_MShef4_J8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961171/SRR7961171_1.fastq.gz -o SRR7961171_GSM3415735_Subclone_J13_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961171/SRR7961171_2.fastq.gz -o SRR7961171_GSM3415735_Subclone_J13_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961172/SRR7961172_1.fastq.gz -o SRR7961172_GSM3415736_Subclone_J15_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961172/SRR7961172_2.fastq.gz -o SRR7961172_GSM3415736_Subclone_J15_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961173/SRR7961173_1.fastq.gz -o SRR7961173_GSM3415737_Subclone_J1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961173/SRR7961173_2.fastq.gz -o SRR7961173_GSM3415737_Subclone_J1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961174/SRR7961174_1.fastq.gz -o SRR7961174_GSM3415738_Subclone_J20_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961174/SRR7961174_2.fastq.gz -o SRR7961174_GSM3415738_Subclone_J20_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961175/SRR7961175_1.fastq.gz -o SRR7961175_GSM3415739_Subclone_J5_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961175/SRR7961175_2.fastq.gz -o SRR7961175_GSM3415739_Subclone_J5_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961178/SRR7961178_1.fastq.gz -o SRR7961178_GSM3415742_Subclone_N10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961178/SRR7961178_2.fastq.gz -o SRR7961178_GSM3415742_Subclone_N10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961176/SRR7961176_1.fastq.gz -o SRR7961176_GSM3415740_Subclone_J6_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961176/SRR7961176_2.fastq.gz -o SRR7961176_GSM3415740_Subclone_J6_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961179/SRR7961179_1.fastq.gz -o SRR7961179_GSM3415743_Subclone_N7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961179/SRR7961179_2.fastq.gz -o SRR7961179_GSM3415743_Subclone_N7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961177/SRR7961177_1.fastq.gz -o SRR7961177_GSM3415741_Subclone_J8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961177/SRR7961177_2.fastq.gz -o SRR7961177_GSM3415741_Subclone_J8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961180/SRR7961180_1.fastq.gz -o SRR7961180_GSM3415744_Subclone_N8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961180/SRR7961180_2.fastq.gz -o SRR7961180_GSM3415744_Subclone_N8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961181/SRR7961181_1.fastq.gz -o SRR7961181_GSM3415745_Subclone_O10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961181/SRR7961181_2.fastq.gz -o SRR7961181_GSM3415745_Subclone_O10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961182/SRR7961182_1.fastq.gz -o SRR7961182_GSM3415746_Subclone_O4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961182/SRR7961182_2.fastq.gz -o SRR7961182_GSM3415746_Subclone_O4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961183/SRR7961183_1.fastq.gz -o SRR7961183_GSM3415747_Subclone_O7_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961183/SRR7961183_2.fastq.gz -o SRR7961183_GSM3415747_Subclone_O7_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961184/SRR7961184_1.fastq.gz -o SRR7961184_GSM3415748_Subclone_P2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961184/SRR7961184_2.fastq.gz -o SRR7961184_GSM3415748_Subclone_P2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961185/SRR7961185_1.fastq.gz -o SRR7961185_GSM3415749_Subclone_P4_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/005/SRR7961185/SRR7961185_2.fastq.gz -o SRR7961185_GSM3415749_Subclone_P4_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961186/SRR7961186_1.fastq.gz -o SRR7961186_GSM3415750_Subclone_P8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/006/SRR7961186/SRR7961186_2.fastq.gz -o SRR7961186_GSM3415750_Subclone_P8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961187/SRR7961187_1.fastq.gz -o SRR7961187_GSM3415751_Subclone_Q10_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/007/SRR7961187/SRR7961187_2.fastq.gz -o SRR7961187_GSM3415751_Subclone_Q10_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961188/SRR7961188_1.fastq.gz -o SRR7961188_GSM3415752_Subclone_Q12_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/008/SRR7961188/SRR7961188_2.fastq.gz -o SRR7961188_GSM3415752_Subclone_Q12_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961189/SRR7961189_1.fastq.gz -o SRR7961189_GSM3415753_Subclone_Q8_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961189/SRR7961189_2.fastq.gz -o SRR7961189_GSM3415753_Subclone_Q8_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz ```

ewels commented 3 years ago

That is certainly a lot of data 👍🏻

Downloading now, will see how big it is. Might be a little excessive, but I like it 😅

ewels commented 3 years ago

You managed to fill my project partition on the cluster 😆 Trying again now in a different project..

drpatelh commented 3 years ago

Definitely excessive 😅 This has to be browsable on the website!

FelixKrueger commented 3 years ago

Well - sorry - I thought you wanted it excessive... Are you trying an entirely different project, or just a subset?

ewels commented 3 years ago

Yeah good point about the website. Maybe 108 samples is a bit much 😅

Do you think we could pick maybe 3 or 4 conditions each with 2-3 replicates from this bunch? So ~9-12 samples total? Not very excessive after all really. But maybe a bit more useful to browse.

drpatelh commented 3 years ago

Yup, subset would be great. Maybe we should show you an example if you haven't seen one already. Have a butchers here

FelixKrueger commented 3 years ago

I suppose one would have to change the mega in megatest, but how about something like this:

download.sh

```bash #!/usr/bin/env bash curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961089/SRR7961089_1.fastq.gz -o SRR7961089_GSM3415653_MShef11_bulk_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/009/SRR7961089/SRR7961089_2.fastq.gz -o SRR7961089_GSM3415653_MShef11_bulk_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961103/SRR7961103_1.fastq.gz -o SRR7961103_GSM3415667_MShef11_low_oxygen_Q2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961103/SRR7961103_2.fastq.gz -o SRR7961103_GSM3415667_MShef11_low_oxygen_Q2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961102/SRR7961102_1.fastq.gz -o SRR7961102_GSM3415666_MShef11_low_oxygen_Q1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/002/SRR7961102/SRR7961102_2.fastq.gz -o SRR7961102_GSM3415666_MShef11_low_oxygen_Q1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961104/SRR7961104_1.fastq.gz -o SRR7961104_GSM3415668_MShef11_low_oxygen_Q3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961104/SRR7961104_2.fastq.gz -o SRR7961104_GSM3415668_MShef11_low_oxygen_Q3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961150/SRR7961150_1.fastq.gz -o SRR7961150_GSM3415714_MShef4_bulk_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/000/SRR7961150/SRR7961150_2.fastq.gz -o SRR7961150_GSM3415714_MShef4_bulk_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961161/SRR7961161_1.fastq.gz -o SRR7961161_GSM3415725_MShef4_J1_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/001/SRR7961161/SRR7961161_2.fastq.gz -o SRR7961161_GSM3415725_MShef4_J1_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961164/SRR7961164_1.fastq.gz -o SRR7961164_GSM3415728_MShef4_J3_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/004/SRR7961164/SRR7961164_2.fastq.gz -o SRR7961164_GSM3415728_MShef4_J3_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961163/SRR7961163_1.fastq.gz -o SRR7961163_GSM3415727_MShef4_J2_BSseq_Homo_sapiens_Bisulfite-Seq_1.fastq.gz curl -L ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR796/003/SRR7961163/SRR7961163_2.fastq.gz -o SRR7961163_GSM3415727_MShef4_J2_BSseq_Homo_sapiens_Bisulfite-Seq_2.fastq.gz ```

2 cell lines as bulk sample, and one sub-condition with 3 replicates each?

ewels commented 3 years ago

Downloading now 👍🏻

ewels commented 3 years ago

ok, files synced - you can view them here: https://nf-co.re/methylseq/dev/results#methylseq/input_data/

Hopefully that looks about right in terms of file sizes etc! I'll try to put together test_full.config for the paths when I get a chance.

ewels commented 3 years ago

ok, first run going with the new data: https://tower.nf/watch/PmNwX48oJ17L6

Log in using nf-core-awstests@mailinator.com and get the login email here if you want to see (you can then share the run on to your own account for easier future reference if you want).

ewels commented 3 years ago

Tests now running for the v1.6 release - some issues with the config and run, but the test data all seems to have worked well! I think that we can close this issue.

nf-core / methylseq

Set up AWS megatests #174