search

dereneaton / ipyrad

Interactive assembly and analysis of RAD-seq data sets
http://ipyrad.readthedocs.io
GNU General Public License v3.0
72 stars 39 forks source link

step 7：An error occurred during building vcf file #306

Closed AfterDream closed 4 years ago

AfterDream commented 6 years ago

Step 7: Filter and write output files for 57 Samples [####################] 100% filtering loci | 0:02:18
[####################] 100% building loci/stats | 0:01:56
[# ] 8% building vcf file | 0:01:01
Encountered an unexpected error (see ./ipyrad_log.txt) Error message is below ------------------------------- ValueError(could not broadcast input array from shape (2) into shape (1))

AfterDream commented 6 years ago

sometime I can not quit the ipyrad~And my data sets are pair-end GBS data

isaacovercast commented 6 years ago

Can you run step 7 with the -d flag and post the last 20 or so lines of the ipyrad_log.txt file?

AfterDream commented 6 years ago

I just rerun the step 7 with the -d flag, then get this: 2018-09-07 09:20:25,481 pid=90039 [write_outfiles.py] INFO idx 37 2018-09-07 09:21:19,963 pid=89253 [write_outfiles.py] INFO Writing .vcf file 2018-09-07 09:22:13,290 pid=90223 [write_outfiles.py] INFO acatg.shape (2, 57, 403, 4) 2018-09-07 09:22:13,499 pid=90412 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:15,173 pid=90322 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:15,475 pid=90207 [write_outfiles.py] INFO acatg.shape (2, 57, 403, 4) 2018-09-07 09:22:15,913 pid=90366 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:16,196 pid=90250 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:16,386 pid=90398 [write_outfiles.py] INFO acatg.shape (2, 57, 403, 4) 2018-09-07 09:22:16,922 pid=90223 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:17,108 pid=90265 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:17,783 pid=90317 [write_outfiles.py] INFO acatg.shape (2, 57, 403, 4) 2018-09-07 09:22:17,829 pid=90398 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:18,307 pid=90207 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:18,661 pid=90228 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:19,445 pid=90388 [write_outfiles.py] INFO acatg.shape (3, 57, 403, 4) 2018-09-07 09:22:19,808 pid=90300 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:19,837 pid=90191 [write_outfiles.py] INFO acatg.shape (2, 57, 403, 4) 2018-09-07 09:22:20,062 pid=90247 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:20,694 pid=90317 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:20,769 pid=90092 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:20,937 pid=90336 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:21,144 pid=90288 [write_outfiles.py] INFO acatg.shape (1, 57, 403, 4) 2018-09-07 09:22:21,579 pid=89995 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:21,719 pid=90357 [write_outfiles.py] INFO acatg.shape (2, 57, 403, 4) 2018-09-07 09:22:22,000 pid=89909 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:22,786 pid=89253 [assembly.py] ERROR ValueError(could not broadcast input array from shape (2) into shape (1)) 2018-09-07 09:22:22,881 pid=90191 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:22,952 pid=90153 [write_outfiles.py] INFO acatg.shape (1, 57, 403, 4) 2018-09-07 09:22:23,110 pid=90376 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:23,584 pid=89882 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:23,846 pid=89646 [write_outfiles.py] INFO acatg.shape (1, 57, 403, 4) 2018-09-07 09:22:23,880 pid=90177 [write_outfiles.py] INFO acatg.shape (0, 57, 403, 4) 2018-09-07 09:22:23,980 pid=89253 [assembly.py] INFO interrupted engine 0 w/ SIGINT to 89303 2018-09-07 09:22:23,990 pid=89253 [assembly.py] INFO interrupted engine 1 w/ SIGINT to 89304 2018-09-07 09:22:23,997 pid=89253 [assembly.py] INFO interrupted engine 2 w/ SIGINT to 89307 2018-09-07 09:22:24,006 pid=89253 [assembly.py] INFO interrupted engine 3 w/ SIGINT to 89324 2018-09-07 09:22:24,014 pid=89253 [assembly.py] INFO interrupted engine 4 w/ SIGINT to 89329 2018-09-07 09:22:24,022 pid=89253 [assembly.py] INFO interrupted engine 5 w/ SIGINT to 89334 2018-09-07 09:22:24,030 pid=89253 [assembly.py] INFO interrupted engine 6 w/ SIGINT to 89339 2018-09-07 09:22:24,037 pid=89253 [assembly.py] INFO interrupted engine 7 w/ SIGINT to 89346 2018-09-07 09:22:24,044 pid=89253 [assembly.py] INFO interrupted engine 8 w/ SIGINT to 89351 2018-09-07 09:22:24,052 pid=89253 [assembly.py] INFO interrupted engine 9 w/ SIGINT to 89363 2018-09-07 09:22:24,060 pid=89253 [assembly.py] INFO interrupted engine 10 w/ SIGINT to 89379 2018-09-07 09:22:24,067 pid=89253 [assembly.py] INFO interrupted engine 11 w/ SIGINT to 89391 2018-09-07 09:22:24,074 pid=89253 [assembly.py] INFO interrupted engine 12 w/ SIGINT to 89404 2018-09-07 09:22:24,082 pid=89253 [assembly.py] INFO interrupted engine 13 w/ SIGINT to 89416 2018-09-07 09:22:24,090 pid=89253 [assembly.py] INFO interrupted engine 14 w/ SIGINT to 89428 2018-09-07 09:22:24,098 pid=89253 [assembly.py] INFO interrupted engine 15 w/ SIGINT to 89438 2018-09-07 09:22:24,106 pid=89253 [assembly.py] INFO interrupted engine 16 w/ SIGINT to 89450 2018-09-07 09:22:24,114 pid=89253 [assembly.py] INFO interrupted engine 17 w/ SIGINT to 89464 2018-09-07 09:22:24,121 pid=89253 [assembly.py] INFO interrupted engine 18 w/ SIGINT to 89474 2018-09-07 09:22:24,128 pid=89253 [assembly.py] INFO interrupted engine 19 w/ SIGINT to 89486 2018-09-07 09:22:24,136 pid=89253 [assembly.py] INFO interrupted engine 20 w/ SIGINT to 89499 2018-09-07 09:22:24,143 pid=89253 [assembly.py] INFO interrupted engine 21 w/ SIGINT to 89511 2018-09-07 09:22:24,151 pid=89253 [assembly.py] INFO interrupted engine 22 w/ SIGINT to 89524 2018-09-07 09:22:24,161 pid=89253 [assembly.py] INFO interrupted engine 23 w/ SIGINT to 89535 2018-09-07 09:22:24,169 pid=89253 [assembly.py] INFO interrupted engine 24 w/ SIGINT to 89547 2018-09-07 09:22:24,177 pid=89253 [assembly.py] INFO interrupted engine 25 w/ SIGINT to 89561 2018-09-07 09:22:24,185 pid=89253 [assembly.py] INFO interrupted engine 26 w/ SIGINT to 89573 2018-09-07 09:22:24,193 pid=89253 [assembly.py] INFO interrupted engine 27 w/ SIGINT to 89583 2018-09-07 09:22:24,200 pid=89253 [assembly.py] INFO interrupted engine 28 w/ SIGINT to 89595 2018-09-07 09:22:24,206 pid=89253 [assembly.py] INFO interrupted engine 29 w/ SIGINT to 89607 2018-09-07 09:22:24,214 pid=89253 [assembly.py] INFO interrupted engine 30 w/ SIGINT to 89621 2018-09-07 09:22:24,224 pid=89253 [assembly.py] INFO interrupted engine 31 w/ SIGINT to 89632 2018-09-07 09:22:24,232 pid=89253 [assembly.py] INFO interrupted engine 32 w/ SIGINT to 89646 2018-09-07 09:22:24,238 pid=89253 [assembly.py] INFO interrupted engine 33 w/ SIGINT to 89656 2018-09-07 09:22:24,245 pid=89253 [assembly.py] INFO interrupted engine 34 w/ SIGINT to 89668 2018-09-07 09:22:24,252 pid=89253 [assembly.py] INFO interrupted engine 35 w/ SIGINT to 89693 2018-09-07 09:22:24,260 pid=89253 [assembly.py] INFO interrupted engine 36 w/ SIGINT to 89680 2018-09-07 09:22:24,267 pid=89253 [assembly.py] INFO interrupted engine 37 w/ SIGINT to 89706 2018-09-07 09:22:24,275 pid=89253 [assembly.py] INFO interrupted engine 38 w/ SIGINT to 89718 2018-09-07 09:22:24,282 pid=89253 [assembly.py] INFO interrupted engine 39 w/ SIGINT to 89728 2018-09-07 09:22:24,290 pid=89253 [assembly.py] INFO interrupted engine 40 w/ SIGINT to 89741 2018-09-07 09:22:24,297 pid=89253 [assembly.py] INFO interrupted engine 41 w/ SIGINT to 89752 2018-09-07 09:22:24,304 pid=89253 [assembly.py] INFO interrupted engine 42 w/ SIGINT to 89766 2018-09-07 09:22:24,312 pid=89253 [assembly.py] INFO interrupted engine 43 w/ SIGINT to 89798 2018-09-07 09:22:24,319 pid=89253 [assembly.py] INFO interrupted engine 44 w/ SIGINT to 89781 2018-09-07 09:22:24,326 pid=89253 [assembly.py] INFO interrupted engine 45 w/ SIGINT to 89802 2018-09-07 09:22:24,333 pid=89253 [assembly.py] INFO interrupted engine 46 w/ SIGINT to 89815 2018-09-07 09:22:24,341 pid=89253 [assembly.py] INFO interrupted engine 47 w/ SIGINT to 89825 2018-09-07 09:22:24,349 pid=89253 [assembly.py] INFO interrupted engine 48 w/ SIGINT to 89844 2018-09-07 09:22:24,358 pid=89253 [assembly.py] INFO interrupted engine 49 w/ SIGINT to 89851 2018-09-07 09:22:24,366 pid=89253 [assembly.py] INFO interrupted engine 50 w/ SIGINT to 89870 2018-09-07 09:22:24,380 pid=89253 [assembly.py] INFO interrupted engine 52 w/ SIGINT to 89893 2018-09-07 09:22:24,386 pid=89253 [assembly.py] INFO interrupted engine 53 w/ SIGINT to 89897 2018-09-07 09:22:24,394 pid=89253 [assembly.py] INFO interrupted engine 54 w/ SIGINT to 89909 2018-09-07 09:22:24,402 pid=89253 [assembly.py] INFO interrupted engine 55 w/ SIGINT to 89943 2018-09-07 09:22:24,409 pid=89253 [assembly.py] INFO interrupted engine 56 w/ SIGINT to 89931 2018-09-07 09:22:24,416 pid=89253 [assembly.py] INFO interrupted engine 57 w/ SIGINT to 89946 2018-09-07 09:22:24,423 pid=89253 [assembly.py] INFO interrupted engine 58 w/ SIGINT to 89960 2018-09-07 09:22:24,430 pid=89253 [assembly.py] INFO interrupted engine 59 w/ SIGINT to 89970 2018-09-07 09:22:24,437 pid=89253 [assembly.py] INFO interrupted engine 60 w/ SIGINT to 89983 2018-09-07 09:22:24,445 pid=89253 [assembly.py] INFO interrupted engine 61 w/ SIGINT to 89995 2018-09-07 09:22:24,452 pid=89253 [assembly.py] INFO interrupted engine 62 w/ SIGINT to 90010 2018-09-07 09:22:24,459 pid=89253 [assembly.py] INFO interrupted engine 63 w/ SIGINT to 90027 2018-09-07 09:22:24,466 pid=89253 [assembly.py] INFO interrupted engine 64 w/ SIGINT to 90039 2018-09-07 09:22:24,473 pid=89253 [assembly.py] INFO interrupted engine 65 w/ SIGINT to 90051 2018-09-07 09:22:24,481 pid=89253 [assembly.py] INFO interrupted engine 66 w/ SIGINT to 90054 2018-09-07 09:22:24,485 pid=90357 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:24,488 pid=89253 [assembly.py] INFO interrupted engine 67 w/ SIGINT to 90068 2018-09-07 09:22:24,495 pid=89253 [assembly.py] INFO interrupted engine 68 w/ SIGINT to 90080 2018-09-07 09:22:24,502 pid=89253 [assembly.py] INFO interrupted engine 69 w/ SIGINT to 90092 2018-09-07 09:22:24,505 pid=90153 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:24,513 pid=89253 [assembly.py] INFO interrupted engine 70 w/ SIGINT to 90107 2018-09-07 09:22:24,529 pid=89253 [assembly.py] INFO interrupted engine 71 w/ SIGINT to 90123 2018-09-07 09:22:24,539 pid=89253 [assembly.py] INFO interrupted engine 72 w/ SIGINT to 90131 2018-09-07 09:22:24,551 pid=89253 [assembly.py] INFO interrupted engine 73 w/ SIGINT to 90143 2018-09-07 09:22:24,568 pid=89253 [assembly.py] INFO interrupted engine 75 w/ SIGINT to 90165 2018-09-07 09:22:24,603 pid=89253 [assembly.py] INFO interrupted engine 78 w/ SIGINT to 90207 2018-09-07 09:22:24,614 pid=89253 [assembly.py] INFO interrupted engine 79 w/ SIGINT to 90223 2018-09-07 09:22:24,628 pid=89253 [assembly.py] INFO interrupted engine 80 w/ SIGINT to 90228 2018-09-07 09:22:24,643 pid=89253 [assembly.py] INFO interrupted engine 81 w/ SIGINT to 90247 2018-09-07 09:22:24,663 pid=89253 [assembly.py] INFO interrupted engine 82 w/ SIGINT to 90250 2018-09-07 09:22:24,679 pid=89253 [assembly.py] INFO interrupted engine 83 w/ SIGINT to 90265 2018-09-07 09:22:24,690 pid=89253 [assembly.py] INFO interrupted engine 84 w/ SIGINT to 90281 2018-09-07 09:22:24,700 pid=89253 [assembly.py] INFO interrupted engine 85 w/ SIGINT to 90288 2018-09-07 09:22:24,708 pid=89253 [assembly.py] INFO interrupted engine 86 w/ SIGINT to 90300 2018-09-07 09:22:24,716 pid=89253 [assembly.py] INFO interrupted engine 87 w/ SIGINT to 90322 2018-09-07 09:22:24,724 pid=89253 [assembly.py] INFO interrupted engine 88 w/ SIGINT to 90317 2018-09-07 09:22:24,732 pid=89253 [assembly.py] INFO interrupted engine 89 w/ SIGINT to 90336 2018-09-07 09:22:24,750 pid=89253 [assembly.py] INFO interrupted engine 91 w/ SIGINT to 90366 2018-09-07 09:22:24,768 pid=89253 [assembly.py] INFO interrupted engine 93 w/ SIGINT to 90388 2018-09-07 09:22:24,775 pid=89253 [assembly.py] INFO interrupted engine 94 w/ SIGINT to 90398 2018-09-07 09:22:24,783 pid=89253 [assembly.py] INFO interrupted engine 95 w/ SIGINT to 90412 2018-09-07 09:22:25,230 pid=89646 [write_outfiles.py] DEBUG Writing data to vcf 2018-09-07 09:22:25,784 pid=89253 [assembly.py] INFO shutting down engines 2018-09-07 09:22:49,826 pid=89253 [assembly.py] INFO finished shutdown 2018-09-07 09:22:49,831 pid=89253 [__init__.py] INFO debugging turned off

isaacovercast commented 6 years ago

Can you post your params file?

The format of the catgs array is (nloci, len(samples), maxlen, 4) and you're getting values for nloci that look wrong, like 0 or 1 or 2, so there might be some bad filtering being applied at step 7. Please post your params file so I can see your settings.

AfterDream commented 6 years ago

Thanks so much for answering my question. The params: ------- ipyrad params file (v.0.7.27)------------------------------------------- NZL_03 ## [0] [assembly_name]: Assembly name. Used to name output directories for assembly steps ./ ## [1] [project_dir]: Project dir (made in curdir if not present)

[2] [raw_fastq_path]: Location of raw non-demultiplexed fastq files

                           ## [3] [barcodes_path]: Location of barcodes file

/home/mengr/work/05-Zhangchen/data/*.fastq ## [4] [sorted_fastq_path]: Location of demultiplexed/sorted fastq files denovo ## [5] [assembly_method]: Assembly method (denovo, reference, denovo+reference, denovo-reference)

[6] [reference_sequence]: Location of reference sequence file

pairgbs ## [7] [datatype]: Datatype (see docs): rad, gbs, ddrad, etc.

[8] [restriction_overhang]: Restriction overhang (cut1,) or (cut1, cut2)

4 ## [9] [max_low_qual_bases]: Max low quality base calls (Q<20) in a read 33 ## [10] [phred_Qscore_offset]: phred Q score offset (33 is default and very standard) 6 ## [11] [mindepth_statistical]: Min depth for statistical base calling 6 ## [12] [mindepth_majrule]: Min depth for majority-rule base calling 10000 ## [13] [maxdepth]: Max cluster depth within samples 0.90 ## [14] [clust_threshold]: Clustering threshold for de novo assembly 0 ## [15] [max_barcode_mismatch]: Max number of allowable mismatches in barcodes 2 ## [16] [filter_adapters]: Filter for adapters/primers (1 or 2=stricter) 40 ## [17] [filter_min_trim_len]: Min length of reads after adapter trim 2 ## [18] [max_alleles_consens]: Max alleles per site in consensus sequences 4, 4 ## [19] [max_Ns_consens]: Max N's (uncalled bases) in consensus (R1, R2) 6, 6 ## [20] [max_Hs_consens]: Max Hs (heterozygotes) in consensus (R1, R2) 35 ## [21] [min_samples_locus]: Min # samples per locus for output 20, 20 ## [22] [max_SNPs_locus]: Max # SNPs per locus (R1, R2) 5, 5 ## [23] [max_Indels_locus]: Max # of indels per locus (R1, R2) 0.5 ## [24] [max_shared_Hs_locus]: Max # heterozygous sites per locus (R1, R2) 0, 0 ## [25] [trim_reads]: Trim raw read edges (R1>, <R1, R2>, <R2) (see docs) 0, 0 ## [26] [trim_loci]: Trim locus edges (see docs) (R1>, <R1, R2>, <R2) p, s, v, n, k ## [27] [output_formats]: Output formats (see docs)

[28] [pop_assign_file]: Path to population assignment file

isaacovercast commented 6 years ago

Hm, ok. Can I see the contents of the _stats.txt file in the _outfiles directory? This should help.

AfterDream commented 6 years ago

This is the hole stats.txt file:

The number of loci caught by each filter.

ipyrad API location: [assembly].stats_dfs.s7_filters

                        total_filters  applied_order  retained_loci

total_prefiltered_loci 1011458 0 1011458 filtered_by_rm_duplicates 30319 30319 981139 filtered_by_max_indels 1490 1490 979649 filtered_by_max_snps 3241 469 979180 filtered_by_max_shared_het 2263 668 978512 filtered_by_min_sample 1007153 978054 458 filtered_by_max_alleles 191278 176 282 total_filtered_loci 282 0 282

The number of loci recovered for each Sample.

ipyrad API location: [assembly].stats_dfs.s7_samples

       sample_coverage

NZL_01raw 181 NZL_02raw 189 NZL_03raw 221 NZL_04raw 226 NZL_05raw 244 NZL_06raw 230 NZL_07raw 247 NZL_08raw 213 NZL_10raw 207 NZL_11raw 220 NZL_12raw 229 NZL_15raw 170 NZL_16raw 247 NZL_17raw 225 NZL_18raw 244 NZL_19raw 182 NZL_20raw 198 NZL_21raw 179 NZL_23raw 221 NZL_24raw 214 NZL_25raw 241 NZL_26raw 240 NZL_28raw 221 NZL_29raw 212 NZL_30raw 206 NZL_31raw 201 NZL_32raw 216 NZL_33raw 185 NZL_34raw 221 NZL_36raw 210 NZL_37raw 211 NZL_38raw 220 NZL_40raw 222 NZL_41raw 164 NZL_44raw 233 NZL_45raw 242 NZL_46raw 232 NZL_47raw 225 NZL_48raw 240 NZL_49raw 247 NZL_50raw 250 NZL_51raw 257 NZL_52raw 234 NZL_53raw 234 NZL_55raw 242 NZL_56raw 248 NZL_59raw 201 NZL_61raw 196 NZL_63raw 246 NZL_64raw 227 NZL_65raw 238 NZL_67raw 220 NZL_68raw 231 NZL_69raw 233 NZL_70raw 239 NZL_74raw 213 NZL_75raw 236

The number of loci for which N taxa have data.

ipyrad API location: [assembly].stats_dfs.s7_loci

locus_coverage  sum_coverage

1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 10 0 0 11 0 0 12 0 0 13 0 0 14 0 0 15 0 0 16 0 0 17 0 0 18 0 0 19 0 0 20 0 0 21 0 0 22 0 0 23 0 0 24 0 0 25 0 0 26 0 0 27 0 0 28 0 0 29 0 0 30 0 0 31 0 0 32 0 0 33 0 0 34 0 0 35 17 17 36 19 36 37 15 51 38 15 66 39 9 75 40 17 92 41 15 107 42 11 118 43 8 126 44 14 140 45 13 153 46 15 168 47 13 181 48 10 191 49 16 207 50 10 217 51 15 232 52 7 239 53 10 249 54 9 258 55 4 262 56 6 268 57 14 282

The distribution of SNPs (var and pis) per locus.

var = Number of loci with n variable sites (pis + autapomorphies)

pis = Number of loci with n parsimony informative site (minor allele in >1 sample)

ipyrad API location: [assembly].stats_dfs.s7_snps

var  sum_var  pis  sum_pis

0 112 0 200 0 1 28 28 46 46 2 36 100 22 90 3 21 163 6 108 4 16 227 3 120 5 12 287 5 145 6 9 341 0 145 7 7 390 0 145 8 12 486 0 145 9 6 540 0 145 10 4 580 0 145 11 2 602 0 145 12 4 650 0 145 13 5 715 0 145 14 2 743 0 145 15 3 788 0 145 16 1 804 0 145 17 1 821 0 145 18 1 839 0 145

Final Sample stats summary

       state  reads_raw  reads_passed_filter  clusters_total  clusters_hidepth  hetero_est  error_est  reads_consens  loci_in_assembly

NZL_01raw 7 3999049 3978772 1765912 41979 0.031614 0.012781 26611 181 NZL_02raw 7 13091551 13029454 2807595 381788 0.016110 0.005269 325255 189 NZL_03raw 7 12250487 12193396 2836873 331220 0.018161 0.005912 276252 221 NZL_04raw 7 10556203 10508391 2800949 254900 0.018574 0.006326 208304 226 NZL_05raw 7 15792466 15720481 3806523 436335 0.015768 0.005463 373592 244 NZL_06raw 7 16396669 16317633 3203389 504095 0.014332 0.004785 440103 230 NZL_07raw 7 10789059 10738049 3064546 243132 0.019792 0.006643 195262 247 NZL_08raw 7 12548903 12490195 3227314 309662 0.019829 0.006375 253189 213 NZL_10raw 7 4337908 4316017 2197344 40117 0.032313 0.013355 23484 207 NZL_11raw 7 4179779 4160221 1699036 53193 0.029621 0.011385 38893 220 NZL_12raw 7 4369406 4343544 1766936 50696 0.030904 0.012769 35195 229 NZL_15raw 7 4054938 4034417 2039995 39177 0.032588 0.013412 22790 170 NZL_16raw 7 10370884 10322446 4022779 190713 0.022831 0.007890 145002 247 NZL_17raw 7 4429516 4406540 1815033 51734 0.030727 0.012543 35952 225 NZL_18raw 7 4461458 4436464 1880812 54867 0.030482 0.012369 38584 244 NZL_19raw 7 3663824 3645446 1797709 32592 0.033324 0.013654 18283 182 NZL_20raw 7 4106069 4086522 1794850 45392 0.031941 0.012758 30065 198 NZL_21raw 7 10872207 10820655 2864425 252687 0.022044 0.007011 201004 179 NZL_23raw 7 11030681 10982146 2827429 264154 0.022380 0.007021 210558 221 NZL_24raw 7 10389735 10345526 2810330 236622 0.022950 0.007167 187062 214 NZL_25raw 7 16344832 16275482 3662248 473183 0.018196 0.005145 406360 241 NZL_26raw 7 14095896 14032357 3042975 382015 0.020550 0.006038 315937 240 NZL_28raw 7 4426561 4392322 2030923 41689 0.031806 0.013729 27153 221 NZL_29raw 7 4530280 4506876 1924863 41643 0.033275 0.013161 24799 212 NZL_30raw 7 3891127 3871757 1876531 35191 0.033322 0.013943 20379 206 NZL_31raw 7 12959086 12900895 3241944 328467 0.020027 0.006276 268542 201 NZL_32raw 7 8292978 8252251 2541148 146708 0.025940 0.008797 107087 216 NZL_33raw 7 9846832 9796707 2949129 189298 0.024933 0.008258 142882 185 NZL_34raw 7 4712512 4691698 2155034 50637 0.031428 0.012491 31449 221 NZL_36raw 7 4351177 4306569 1647751 51646 0.030978 0.013037 36930 210 NZL_37raw 7 4622782 4601329 2014701 55338 0.031653 0.013182 37703 211 NZL_38raw 7 3927794 3902169 1630933 44763 0.032034 0.012915 30677 220 NZL_40raw 7 5878144 5844654 2770191 73541 0.025235 0.010788 50204 222 NZL_41raw 7 4183911 4156492 2177265 39311 0.029347 0.013254 23786 164 NZL_44raw 7 17376367 17262345 4540881 387329 0.018593 0.005943 320095 233 NZL_45raw 7 5340661 5310053 2592788 60012 0.024445 0.010750 41212 242 NZL_46raw 7 7316828 7270769 3466758 90928 0.024233 0.009971 63497 232 NZL_47raw 7 4215231 4189018 2196561 42553 0.028033 0.012043 27027 225 NZL_48raw 7 4312757 4288300 2086750 42888 0.029932 0.012976 28172 240 NZL_49raw 7 5271965 5241084 2660040 58285 0.026690 0.011244 38484 247 NZL_50raw 7 6621742 6589585 2543222 84622 0.030227 0.011054 56189 250 NZL_51raw 7 8285679 8241749 3539285 122418 0.022906 0.008699 90114 257 NZL_52raw 7 5689618 5657411 2655263 68941 0.024044 0.010326 47745 234 NZL_53raw 7 12077983 12013835 4082565 240498 0.015277 0.006031 197647 234 NZL_55raw 7 12215104 12146227 4741249 203468 0.021025 0.007618 157688 242 NZL_56raw 7 5156008 5132840 2166315 59525 0.032166 0.012085 37767 248 NZL_59raw 7 5192643 5166038 2510546 45720 0.033531 0.013591 27505 201 NZL_61raw 7 8996225 8954061 2888069 154691 0.026665 0.009080 113902 196 NZL_63raw 7 4582949 4561128 2082146 43961 0.033124 0.013639 26229 246 NZL_64raw 7 6847902 6816260 2455353 97735 0.027585 0.010158 69039 227 NZL_65raw 7 9740897 9696551 3302234 159032 0.026137 0.009117 117542 238 NZL_67raw 7 6228635 6199372 2280758 87525 0.028372 0.010205 60573 220 NZL_68raw 7 4126387 4107063 1966719 38408 0.034969 0.013654 22449 231 NZL_69raw 7 7635219 7600839 2715525 117919 0.027125 0.009742 84375 233 NZL_70raw 7 8083518 8048730 2742910 135633 0.022382 0.007955 103261 239 NZL_74raw 7 11088641 11041884 3004591 236956 0.023063 0.007496 184989 213 NZL_75raw 7 7571906 7536468 2673077 113380 0.027701 0.009933 79882 236

isaacovercast commented 6 years ago

Well, your filter min sample and max allele values are probably causing the problem. You have 1e6 loci in the dataset and are ending up with 282. You should try relaxing the min sample value substantially, also the clustering threshold value could be too low sitll.

filtered_by_min_sample 1007153 978054 458
filtered_by_max_alleles 191278 176 282
isaacovercast commented 4 years ago

Closing stale issue. This is also almost certainly fixed in the new version, v.0.9