Paired end read option in Mageck not working

Hi, thank you so much for developing this fantastic MAGeCK package, its really helpful in analysing CRISPR-screen data. However, I am facing issues when I try to analyse the data using the paired end read sequencing (--fastq-2) option.

I have detailed the problem below :-

This is the code I use to to run MAGeCK :

source activate mageck-vispr

cd into/the/directory_that_contains_library.csv_and_all_the_.fastq.gz_files

mageck count -l library.txt -n AML_CRISPR_Screen_SJM --sample-label cd11bc1,cd11bc2,cd11bc3,toc1,toc2,toc3 --fastq cd11bc1_IGO_12933_4_S218_L004_R1_001.fastq.gz cd11bc2_IGO_12933_5_S219_L004_R1_001.fastq.gz cd11bc3_IGO_12933_6_S220_L004_R1_001.fastq.gz toc1_IGO_12933_1_S215_L004_R1_001.fastq.gz toc2_IGO_12933_2_S216_L004_R1_001.fastq.gz toc3_IGO_12933_3_S217_L004_R1_001.fastq.gz --fastq-2 cd11bc1_IGO_12933_4_S218_L004_R2_001.fastq.gz cd11bc2_IGO_12933_5_S219_L004_R2_001.fastq.gz cd11bc3_IGO_12933_6_S220_L004_R2_001.fastq.gz toc1_IGO_12933_1_S215_L004_R2_001.fastq.gz toc2_IGO_12933_2_S216_L004_R2_001.fastq.gz toc3_IGO_12933_3_S217_L004_R2_001.fastq.gz

This is the output I get (error message is towards the end).

INFO @ Mon, 12 Sep 2022 13:26:51: Parameters: /Users/soumyasharma/opt/anaconda3/envs/mageckenv/bin/mageck count -l library.txt -n AML_CRISPR_Screen_SJM --sample-label cd11bc1,cd11bc2,cd11bc3,toc1,toc2,toc3 --fastq cd11bc1_IGO_12933_4_S218_L004_R1_001.fastq.gz cd11bc2_IGO_12933_5_S219_L004_R1_001.fastq.gz cd11bc3_IGO_12933_6_S220_L004_R1_001.fastq.gz toc1_IGO_12933_1_S215_L004_R1_001.fastq.gz toc2_IGO_12933_2_S216_L004_R1_001.fastq.gz toc3_IGO_12933_3_S217_L004_R1_001.fastq.gz --fastq-2 cd11bc1_IGO_12933_4_S218_L004_R2_001.fastq.gz cd11bc2_IGO_12933_5_S219_L004_R2_001.fastq.gz cd11bc3_IGO_12933_6_S220_L004_R2_001.fastq.gz toc1_IGO_12933_1_S215_L004_R2_001.fastq.gz toc2_IGO_12933_2_S216_L004_R2_001.fastq.gz toc3_IGO_12933_3_S217_L004_R2_001.fastq.gz INFO @ Mon, 12 Sep 2022 13:26:51: Welcome to MAGeCK v0.5.9.5. Command: count INFO @ Mon, 12 Sep 2022 13:26:51: Header line of the library file detected; skip the first line ... INFO @ Mon, 12 Sep 2022 13:26:51: Loading 13474 predefined sgRNAs. WARNING @ Mon, 12 Sep 2022 13:26:51: There are 0 sgRNAs with duplicated sequences. INFO @ Mon, 12 Sep 2022 13:26:51: Parsing FASTQ file cd11bc1_IGO_12933_4_S218_L004_R1_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:26:51: Determining the trim-5 length of FASTQ file cd11bc1_IGO_12933_4_S218_L004_R1_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:26:51: Possible gRNA lengths:20 INFO @ Mon, 12 Sep 2022 13:26:51: Processing 0M reads ... INFO @ Mon, 12 Sep 2022 13:26:53: Read length:101 INFO @ Mon, 12 Sep 2022 13:26:53: Total tested reads: 100001, mapped: 23399(0.23398766012339878) WARNING @ Mon, 12 Sep 2022 13:26:53: Only 23.398766012339877 % of the reads can be identified.
INFO @ Mon, 12 Sep 2022 13:26:53: --trim-5 test data: (trim_length reads fraction) INFO @ Mon, 12 Sep 2022 13:26:53: 21 21217 0.9067481516304116 INFO @ Mon, 12 Sep 2022 13:26:53: 20 647 0.027650754305739562 INFO @ Mon, 12 Sep 2022 13:26:53: 19 202 0.008632847557587931 INFO @ Mon, 12 Sep 2022 13:26:53: 18 201 0.008590110688490962 INFO @ Mon, 12 Sep 2022 13:26:53: 17 199 0.008504636950297021 INFO @ Mon, 12 Sep 2022 13:26:53: 16 138 0.005897687935381854 INFO @ Mon, 12 Sep 2022 13:26:53: 14 136 0.005812214197187914 INFO @ Mon, 12 Sep 2022 13:26:53: 13 113 0.004829266207957605 INFO @ Mon, 12 Sep 2022 13:26:53: 12 112 0.004786529338860635 INFO @ Mon, 12 Sep 2022 13:26:53: 15 105 0.004487371255181846 INFO @ Mon, 12 Sep 2022 13:26:53: 22 73 0.003119791444078807 INFO @ Mon, 12 Sep 2022 13:26:53: 11 63 0.002692422753109107 INFO @ Mon, 12 Sep 2022 13:26:53: 8 46 0.001965895978460618 INFO @ Mon, 12 Sep 2022 13:26:53: 7 34 0.0014530535492969785 INFO @ Mon, 12 Sep 2022 13:26:53: 10 34 0.0014530535492969785 INFO @ Mon, 12 Sep 2022 13:26:53: 9 19 0.0008120005128424291 INFO @ Mon, 12 Sep 2022 13:26:53: 1 13 0.0005555792982606094 INFO @ Mon, 12 Sep 2022 13:26:53: 6 9 0.0003846318218727296 INFO @ Mon, 12 Sep 2022 13:26:53: 3 8 0.00034189495277575963 INFO @ Mon, 12 Sep 2022 13:26:53: 4 7 0.0002991580836787897 INFO @ Mon, 12 Sep 2022 13:26:53: 0 7 0.0002991580836787897 INFO @ Mon, 12 Sep 2022 13:26:53: 2 7 0.0002991580836787897 INFO @ Mon, 12 Sep 2022 13:26:53: 5 3 0.00012821060729090986 INFO @ Mon, 12 Sep 2022 13:26:53: 25 2 8.547373819393991e-05 INFO @ Mon, 12 Sep 2022 13:26:53: 24 1 4.2736869096969954e-05 INFO @ Mon, 12 Sep 2022 13:26:53: 23 1 4.2736869096969954e-05 INFO @ Mon, 12 Sep 2022 13:26:53: 34 1 4.2736869096969954e-05 INFO @ Mon, 12 Sep 2022 13:26:53: 45 1 4.2736869096969954e-05 INFO @ Mon, 12 Sep 2022 13:26:53: Auto determination of trim5 results: 21 INFO @ Mon, 12 Sep 2022 13:26:53: Determining the trim-5 length of FASTQ file cd11bc1_IGO_12933_4_S218_L004_R2_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:26:53: Possible gRNA lengths:20 INFO @ Mon, 12 Sep 2022 13:26:53: Processing 0M reads ... INFO @ Mon, 12 Sep 2022 13:26:59: Read length:101 INFO @ Mon, 12 Sep 2022 13:26:59: Total tested reads: 100001, mapped: 283(0.002829971700282997) WARNING @ Mon, 12 Sep 2022 13:26:59: Only 0.2829971700282997 % of the reads can be identified.
INFO @ Mon, 12 Sep 2022 13:26:59: --trim-5 test data: (trim_length reads fraction) INFO @ Mon, 12 Sep 2022 13:26:59: 20 36 0.127208480565371 INFO @ Mon, 12 Sep 2022 13:26:59: 21 23 0.0812720848056537 INFO @ Mon, 12 Sep 2022 13:26:59: 41 11 0.038869257950530034 INFO @ Mon, 12 Sep 2022 13:26:59: 63 10 0.0353356890459364 INFO @ Mon, 12 Sep 2022 13:26:59: 19 10 0.0353356890459364 INFO @ Mon, 12 Sep 2022 13:26:59: 70 9 0.03180212014134275 INFO @ Mon, 12 Sep 2022 13:26:59: 65 7 0.024734982332155476 INFO @ Mon, 12 Sep 2022 13:26:59: 38 6 0.02120141342756184 INFO @ Mon, 12 Sep 2022 13:26:59: 46 6 0.02120141342756184 INFO @ Mon, 12 Sep 2022 13:26:59: 42 6 0.02120141342756184 INFO @ Mon, 12 Sep 2022 13:26:59: 48 6 0.02120141342756184 INFO @ Mon, 12 Sep 2022 13:26:59: 62 6 0.02120141342756184 INFO @ Mon, 12 Sep 2022 13:26:59: 40 6 0.02120141342756184 INFO @ Mon, 12 Sep 2022 13:26:59: 25 5 0.0176678445229682 INFO @ Mon, 12 Sep 2022 13:26:59: 23 5 0.0176678445229682 INFO @ Mon, 12 Sep 2022 13:26:59: 61 5 0.0176678445229682 INFO @ Mon, 12 Sep 2022 13:26:59: 26 5 0.0176678445229682 INFO @ Mon, 12 Sep 2022 13:26:59: 64 5 0.0176678445229682 INFO @ Mon, 12 Sep 2022 13:26:59: 59 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 5 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 67 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 18 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 39 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 69 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 66 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 36 4 0.014134275618374558 INFO @ Mon, 12 Sep 2022 13:26:59: 81 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 44 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 2 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 3 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 15 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 79 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 73 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 60 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 56 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 72 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 1 3 0.01060070671378092 INFO @ Mon, 12 Sep 2022 13:26:59: 31 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 35 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 53 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 57 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 52 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 55 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 16 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 0 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 75 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 10 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 30 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 37 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 34 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 4 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 71 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 29 2 0.007067137809187279 INFO @ Mon, 12 Sep 2022 13:26:59: 78 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 13 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 76 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 54 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 27 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 7 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 51 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 11 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 77 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 12 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 24 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 28 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 68 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 49 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 47 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 17 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 6 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 9 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: 45 1 0.0035335689045936395 INFO @ Mon, 12 Sep 2022 13:26:59: Auto determination of trim5 results: 20,21,41 INFO @ Mon, 12 Sep 2022 13:26:59: Possible gRNA lengths:20 INFO @ Mon, 12 Sep 2022 13:26:59: Processing 0M reads .. INFO @ Mon, 12 Sep 2022 13:27:05: Processing 1M reads .. INFO @ Mon, 12 Sep 2022 13:27:11: Processing 2M reads .. INFO @ Mon, 12 Sep 2022 13:27:17: Processing 3M reads .. INFO @ Mon, 12 Sep 2022 13:27:23: Processing 4M reads .. INFO @ Mon, 12 Sep 2022 13:27:29: Processing 5M reads .. INFO @ Mon, 12 Sep 2022 13:27:35: Processing 6M reads .. INFO @ Mon, 12 Sep 2022 13:27:41: Processing 7M reads .. INFO @ Mon, 12 Sep 2022 13:27:47: Processing 8M reads .. INFO @ Mon, 12 Sep 2022 13:27:53: Processing 9M reads .. INFO @ Mon, 12 Sep 2022 13:26:53: Determining the trim-5 length of FASTQ file cd11bc1_IGO_12933_4_S218_L004_R2_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:26:53: Possible gRNA lengths:20 INFO @ Mon, 12 Sep 2022 13:26:53: Processing 0M reads ... INFO @ Mon, 12 Sep 2022 13:26:59: Read length:101 INFO @ Mon, 12 Sep 2022 13:26:59: Total tested reads: 100001, mapped: 283(0.002829971700282997) WARNING @ Mon, 12 Sep 2022 13:26:59: Only 0.2829971700282997 % of the reads can be identified.
INFO @ Mon, 12 Sep 2022 13:26:59: --trim-5 test data: (trim_length reads fraction) INFO @ Mon, 12 Sep 2022 13:26:59: 20 36 0.127208480565371 INFO @ Mon, 12 Sep 2022 13:27:59: Processing 10M reads .. INFO @ Mon, 12 Sep 2022 13:28:05: Processing 11M reads .. INFO @ Mon, 12 Sep 2022 13:28:11: Processing 12M reads .. INFO @ Mon, 12 Sep 2022 13:28:18: Processing 13M reads .. INFO @ Mon, 12 Sep 2022 13:28:24: Processing 14M reads .. INFO @ Mon, 12 Sep 2022 13:28:30: Processing 15M reads .. INFO @ Mon, 12 Sep 2022 13:28:33: Total: 15477650. INFO @ Mon, 12 Sep 2022 13:28:33: Mapped: 3341122. INFO @ Mon, 12 Sep 2022 13:28:33: Parsing FASTQ file cd11bc2_IGO_12933_5_S219_L004_R1_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:28:33: Determining the trim-5 length of FASTQ file cd11bc2_IGO_12933_5_S219_L004_R1_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:28:33: Possible gRNA lengths:20 INFO @ Mon, 12 Sep 2022 13:28:33: Processing 0M reads ... INFO @ Mon, 12 Sep 2022 13:28:36: Read length:101 INFO @ Mon, 12 Sep 2022 13:28:36: Total tested reads: 100001, mapped: 22696(0.22695773042269576) WARNING @ Mon, 12 Sep 2022 13:28:36: Only 22.695773042269575 % of the reads can be identified.
INFO @ Mon, 12 Sep 2022 13:28:36: --trim-5 test data: (trim_length reads fraction) INFO @ Mon, 12 Sep 2022 13:28:36: 21 21050 0.9274762072611914 INFO @ Mon, 12 Sep 2022 13:28:36: 20 717 0.03159146986253084 INFO @ Mon, 12 Sep 2022 13:28:36: 19 147 0.006476912231230173 INFO @ Mon, 12 Sep 2022 13:28:36: 18 145 0.006388790976383504 INFO @ Mon, 12 Sep 2022 13:28:36: 17 116 0.005111032781106803 INFO @ Mon, 12 Sep 2022 13:28:36: 14 105 0.004626365879450123 INFO @ Mon, 12 Sep 2022 13:28:36: 16 100 0.0044060627423334504 INFO @ Mon, 12 Sep 2022 13:28:36: 22 92 0.0040535777229467745 INFO @ Mon, 12 Sep 2022 13:28:36: 12 61 0.002687698272823405 INFO @ Mon, 12 Sep 2022 13:28:36: 15 57 0.002511455763130067 INFO @ Mon, 12 Sep 2022 13:28:36: 13 56 0.0024673951357067326 INFO @ Mon, 12 Sep 2022 13:28:36: 11 12 0.0005287275290800141 INFO @ Mon, 12 Sep 2022 13:28:36: 8 6 0.00026436376454000703 INFO @ Mon, 12 Sep 2022 13:28:36: 10 5 0.00022030313711667254 INFO @ Mon, 12 Sep 2022 13:28:36: 6 3 0.00013218188227000352 INFO @ Mon, 12 Sep 2022 13:28:36: 0 3 0.00013218188227000352 INFO @ Mon, 12 Sep 2022 13:28:36: 24 2 8.812125484666901e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 1 2 8.812125484666901e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 9 2 8.812125484666901e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 7 2 8.812125484666901e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 3 2 8.812125484666901e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 5 2 8.812125484666901e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 23 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 31 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 68 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 30 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 53 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 2 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 26 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 45 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: 47 1 4.406062742333451e-05 INFO @ Mon, 12 Sep 2022 13:28:36: Auto determination of trim5 results: 21 INFO @ Mon, 12 Sep 2022 13:28:36: Determining the trim-5 length of FASTQ file cd11bc2_IGO_12933_5_S219_L004_R2_001.fastq.gz... INFO @ Mon, 12 Sep 2022 13:28:36: Possible gRNA lengths:20 INFO @ Mon, 12 Sep 2022 13:28:36: Processing 0M reads ... INFO @ Mon, 12 Sep 2022 13:28:42: Read length:101 INFO @ Mon, 12 Sep 2022 13:28:42: Total tested reads: 100001, mapped: 79(0.0007899921000789992) ERROR @ Mon, 12 Sep 2022 13:28:42: Cannot automatically determine the --trim-5 length. Only 0.07899921000789992 % of the reads can be identified.
(mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:53: Determining the trim-5 length of FASTQ file cd11bc1_IGO_12933_4_S218_L004_R2_001.fastq.gz... info: No menu item '@' in node '(dir)Top' (mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:53: Possible gRNA lengths:20 info: No menu item '@' in node '(dir)Top' (mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:53: Processing 0M reads ... info: No menu item '@' in node '(dir)Top' (mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:59: Read length:101 info: No menu item '@' in node '(dir)Top' (mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:59: Total tested reads: 100001, mapped: 283(0.002829971700282997) -bash: syntax error near unexpected token (' (mageckenv) MAC303388:fastq soumyasharma$ WARNING @ Mon, 12 Sep 2022 13:26:59: Only 0.2829971700282997 % of the reads can be identified. -bash: WARNING: command not found (mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:59: --trim-5 test data: (trim_length reads fraction) -bash: syntax error near unexpected token(' (mageckenv) MAC303388:fastq soumyasharma$ INFO @ Mon, 12 Sep 2022 13:26:59: 20360.127208480565371 info: No menu item '@' in node '(dir)Top'

Also, when I run using just the R1 fastq.gz files (only using the fastq option and not --fastq-2) using the code below

mageck count -l library.txt -n AML_CRISPR_Screen_SJM --sample-label cd11bc1,cd11bc2,cd11bc3,toc1,toc2,toc3 --fastq cd11bc1_IGO_12933_4_S218_L004_R1_001.fastq.gz cd11bc2_IGO_12933_5_S219_L004_R1_001.fastq.gz cd11bc3_IGO_12933_6_S220_L004_R1_001.fastq.gz toc1_IGO_12933_1_S215_L004_R1_001.fastq.gz toc2_IGO_12933_2_S216_L004_R1_001.fastq.gz toc3_IGO_12933_3_S217_L004_R1_001.fastq.gz

MAGeCK runs without any errors and gives me the expected results.

Can you please help me figure out what's going on when I try to run MAGeCK on the paired-end sequencing reads and why I am getting that error?

Would really appreciate the help!

davidliwei / mageck

Paired end read option in Mageck not working #8