Error while running example

[andrewSharo]

Thanks for developing such a valuable software!

Sadly, when I go to run the example, I get the following error messages (sorry this is so long):

Traceback (most recent call last): File "/redser4/personal/andrew/src/grenepipe/example/.snakemake/scripts/tmpmpp3gy6n.wrapper.py", line 3, in import sys; sys.path.extend(['/home/asharo/miniconda3/envs/snakemake2/lib/python3.9/site-packages']); import pickle; snakemake = pickle.loads(b'\x80\x04\x95\xac\x1a\x00\x00\x00\x00\x00\x00\x8c\x10snakemake.script\x94\x8c\tSnakemake\x94\x93\x94)\x81\x94}\x94(\x8c\x05input\x94\x8c\x0csnakemake.io\x94\x8c\nInputFiles\x94\x93\x94)\x81\x94(\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S2_U1_R1.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S2_U1_R2.fastq.gz\x94e}\x94(\x8c\x06_names\x94}\x94(\x8c\x02r1\x94K\x00N\x86\x94\x8c\x02r2\x94K\x01N\x86\x94u\x8c\x12_allowed_overrides\x94]\x94(\x8c\x05index\x94\x8c\x04sort\x94eh\x15\x8c\tfunctools\x94\x8c\x07partial\x94\x93\x94h\x06\x8c\x19Namedlist._used_attribute\x94\x93\x94\x85\x94R\x94(h\x1b)}\x94\x8c\x05_name\x94h\x15sNt\x94bh\x16h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x16sNt\x94bh\x0fh\nh\x11h\x0bub\x8c\x06output\x94h\x06\x8c\x0bOutputFiles\x94\x93\x94)\x81\x94(\x8c\x17trimmed/S2-1.1.fastq.gz\x94\x8c\x17trimmed/S2-1.2.fastq.gz\x94\x8c trimmed/S2-1.1.unpaired.fastq.gz\x94\x8c trimmed/S2-1.2.unpaired.fastq.gz\x94e}\x94(h\r}\x94(h\x0fK\x00N\x86\x94h\x11K\x01N\x86\x94\x8c\x0br1_unpaired\x94K\x02N\x86\x94\x8c\x0br2_unpaired\x94K\x03N\x86\x94uh\x13]\x94(h\x15h\x16eh\x15h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x15sNt\x94bh\x16h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x16sNt\x94bh\x0fh)h\x11h*h1h+h3h,ub\x8c\x06params\x94h\x06\x8c\x06Params\x94\x93\x94)\x81\x94(\x8c\x00\x94]\x94(\x8c\tLEADING:3\x94\x8c\nTRAILING:3\x94\x8c\x12SLIDINGWINDOW:4:15\x94\x8c\tMINLEN:36\x94ee}\x94(h\r}\x94(\x8c\x05extra\x94K\x00N\x86\x94\x8c\x07trimmer\x94K\x01N\x86\x94uh\x13]\x94(h\x15h\x16eh\x15h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x15sNt\x94bh\x16h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x16sNt\x94bhJhBhLhCub\x8c\twildcards\x94h\x06\x8c\tWildcards\x94\x93\x94)\x81\x94(\x8c\x02S2\x94\x8c\x011\x94e}\x94(h\r}\x94(\x8c\x06sample\x94K\x00N\x86\x94\x8c\x04unit\x94K\x01N\x86\x94uh\x13]\x94(h\x15h\x16eh\x15h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x15sNt\x94bh\x16h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x16sNt\x94b\x8c\x06sample\x94h[\x8c\x04unit\x94h\ub\x8c\x07threads\x94K\x04\x8c\tresources\x94h\x06\x8c\tResources\x94\x93\x94)\x81\x94(K\x04K\x01\x8c\x04/tmp\x94e}\x94(h\r}\x94(\x8c\x06_cores\x94K\x00N\x86\x94\x8c\x06_nodes\x94K\x01N\x86\x94\x8c\x06tmpdir\x94K\x02N\x86\x94uh\x13]\x94(h\x15h\x16eh\x15h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x15sNt\x94bh\x16h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x16sNt\x94bhvK\x04hxK\x01hzhsub\x8c\x03log\x94h\x06\x8c\x03Log\x94\x93\x94)\x81\x94\x8c\x19logs/trimmomatic/S2-1.log\x94a}\x94(h\r}\x94h\x13]\x94(h\x15h\x16eh\x15h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x15sNt\x94bh\x16h\x19h\x1b\x85\x94R\x94(h\x1b)}\x94h\x1fh\x16sNt\x94bub\x8c\x06config\x94}\x94(\x8c\x04data\x94}\x94(\x8c\x07samples\x94\x8c:/redser4/personal/andrew/src/grenepipe/example/samples.tsv\x94\x8c\treference\x94}\x94(\x8c\x06genome\x94\x8c@/redser4/personal/andrew/src/grenepipe/example/TAIR10_chr_all.fa\x94\x8c\x0eknown-variants\x94]\x94uu\x8c\x08settings\x94}\x94(\x8c\rtrimming-tool\x94\x8c\x0btrimmomatic\x94\x8c\x16merge-paired-end-reads\x94\x89\x8c\x0cmapping-tool\x94\x8c\x06bwamem\x94\x8c\x13filter-mapped-reads\x94\x89\x8c\x11remove-duplicates\x94\x88\x8c\x0fduplicates-tool\x94\x8c\x06picard\x94\x8c\x1arecalibrate-base-qualities\x94\x89\x8c\x0ccalling-tool\x94\x8c\x0fhaplotypecaller\x94\x8c\x11contig-group-size\x94K\x00\x8c\x10restrict-regions\x94]\x94\x8c\x06snpeff\x94\x89\x8c\x03vep\x94\x89\x8c\tmapdamage\x94\x89\x8c\x0edamageprofiler\x94\x89\x8c\x07pileups\x94]\x94\x8c\x0ffrequency-table\x94\x89\x8c\x16frequency-table-fields\x94\x8c\x18COV,FREQ,REF_CNT,ALT_CNT\x94u\x8c\x06params\x94}\x94(\x8c\x0eadapterremoval\x94}\x94(\x8c\x07threads\x94K\x04\x8c\x02se\x94hB\x8c\x02pe\x94hBu\x8c\x08cutadapt\x94}\x94(\x8c\x07threads\x94K\x04\x8c\x02se\x94}\x94(\x8c\x08adapters\x94\x8c%-a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC\x94\x8c\x05extra\x94\x8c\x05-q 20\x94u\x8c\x02pe\x94}\x94(\x8c\x08adapters\x94\x8c%-a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC\x94\x8c\x05extra\x94\x8c\x05-q 20\x94uu\x8c\x05fastp\x94}\x94(\x8c\x07threads\x94K\x04\x8c\x02se\x94hB\x8c\x02pe\x94hBu\x8c\x06skewer\x94}\x94(\x8c\x07threads\x94K\x04\x8c\x02se\x94\x8c\n--mode any\x94\x8c\x02pe\x94\x8c\t--mode pe\x94u\x8c\x0btrimmomatic\x94}\x94(\x8c\x07threads\x94K\x06\x8c\x02se\x94}\x94(\x8c\x05extra\x94hB\x8c\x07trimmer\x94]\x94(\x8c\tLEADING:3\x94\x8c\nTRAILING:3\x94\x8c\x12SLIDINGWINDOW:4:15\x94\x8c\tMINLEN:36\x94eu\x8c\x02pe\x94}\x94(\x8c\x05extra\x94hB\x8c\x07trimmer\x94]\x94(hDhEhFhGeuu\x8c\x07bowtie2\x94}\x94(\x8c\x07threads\x94K\x0c\x8c\x05extra\x94hBu\x8c\x06bwaaln\x94}\x94(\x8c\x07threads\x94K\x04\x8c\x05extra\x94hB\x8c\textra-sam\x94hB\x8c\nextra-sort\x94hBu\x8c\x06bwamem\x94}\x94(\x8c\x07threads\x94K\x0c\x8c\x05extra\x94hB\x8c\nextra-sort\x94hBu\x8c\x07bwamem2\x94}\x94(\x8c\x07threads\x94K\x0c\x8c\x05extra\x94hB\x8c\nextra-sort\x94hBu\x8c\x08samtools\x94}\x94(\x8c\x04view\x94\x8c\x04-q 1\x94\x8c\x06pileup\x94\x8c\x08-d 10000\x94\x8c\x05merge\x94hB\x8c\rmerge-threads\x94K\x04u\x8c\x06picard\x94}\x94(\x8c\x0eMarkDuplicates\x94\x8c\x16REMOVE_DUPLICATES=true\x94\x8c\x16CollectMultipleMetrics\x94}\x94(\x8c\x17AlignmentSummaryMetrics\x94\x88\x8c\x17BaseDistributionByCycle\x94\x88\x8c\rGcBiasMetrics\x94\x88\x8c\x11InsertSizeMetrics\x94\x88\x8c\x15QualityByCycleMetrics\x94\x88\x8c\x1fQualityScoreDistributionMetrics\x94\x88\x8c\x13QualityYieldMetrics\x94\x88\x8c\x05extra\x94\x8c]VALIDATION_STRINGENCY=LENIENT METRIC_ACCUMULATION_LEVEL=null METRIC_ACCUMULATION_LEVEL=SAMPLE\x94uu\x8c\x05dedup\x94}\x94\x8c\x05extra\x94\x8c\x02-m\x94s\x8c\x08bcftools\x94}\x94(\x8c\x07threads\x94K\x08\x8c\x07mpileup\x94\x8c\x11--max-depth 10000\x94\x8c\x04call\x94\x8c\x15--multiallelic-caller\x94u\x8c\tfreebayes\x94}\x94(\x8c\x05extra\x94hB\x8c\x07threads\x94K\x08\x8c\x10compress-threads\x94K\x02\x8c\tchunksize\x94J\xa0\x86\x01\x00u\x8c\x04gatk\x94}\x94(\x8c\x10BaseRecalibrator\x94hB\x8c\x0fHaplotypeCaller\x94hB\x8c\x17HaplotypeCaller-threads\x94K\x02\x8c\rGenotypeGVCFs\x94hBu\x8c\x16variantfiltration-hard\x94}\x94(\x8c\x04snvs\x94\x8cPQD < 2.0 || FS > 60.0 || MQ < 40.0 || MQRankSum < -12.5 || ReadPosRankSum < -8.0\x94\x8c\x06indels\x94\x8c0QD < 2.0 || FS > 200.0 || ReadPosRankSum < -20.0\x94u\x8c\x06fastqc\x94}\x94(\x8c\x05input\x94\x8c\x07samples\x94\x8c\x05extra\x94hBu\x8c\x08qualimap\x94}\x94(\x8c\x05extra\x94hB\x8c\x07threads\x94K\x02u\x8c\x06snpeff\x94}\x94(\x8c\x04name\x94\x8c\x14Arabidopsis_thaliana\x94\x8c\x0cdownload-dir\x94hB\x8c\x05extra\x94\x8c\x06-Xmx4g\x94u\x8c\x03vep\x94}\x94(\x8c\x07species\x94\x8c\x14arabidopsis_thaliana\x94\x8c\x05build\x94\x8c\x06TAIR10\x94\x8c\x07release\x94Kh\x8c\tcache-url\x94\x8cCftp://ftp.ebi.ac.uk/ensemblgenomes/pub/plants/current/variation/vep\x94\x8c\x07plugins\x94]\x94\x8c\x07LoFtool\x94a\x8c\tcache-dir\x94hB\x8c\x0bplugins-dir\x94hB\x8c\x05extra\x94hBu\x8c\tmapdamage\x94}\x94\x8c\x05extra\x94hBs\x8c\x0edamageprofiler\x94}\x94\x8c\x05extra\x94hBs\x8c\x07multiqc\x94}\x94\x8c\x05extra\x94hBsu\x8c\x06global\x94}\x94(\x8c\x07samples\x94\x8c\x11pandas.core.frame\x94\x8c\tDataFrame\x94\x93\x94)\x81\x94}\x94(\x8c\x04_mgr\x94\x8c\x1epandas.core.internals.managers\x94\x8c\x0cBlockManager\x94\x93\x94\x8c\x16pandas._libs.internals\x94\x8c\x0f_unpickle_block\x94\x93\x94\x8c\x15numpy.core.multiarray\x94\x8c\x0c_reconstruct\x94\x93\x94\x8c\x05numpy\x94\x8c\x07ndarray\x94\x93\x94K\x00\x85\x94C\x01b\x94\x87\x94R\x94(K\x01K\x05K\x04\x86\x94jp\x01\x00\x00\x8c\x05dtype\x94\x93\x94\x8c\x02O8\x94\x89\x88\x87\x94R\x94(K\x03\x8c\x01|\x94NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK?t\x94b\x89]\x94(\x8c\x02S1\x94\x8c\x02S2\x94j\x81\x01\x00\x00\x8c\x02S3\x94h\h\\x8c\x012\x94h\\x8c\x01-\x94j\x84\x01\x00\x00j\x84\x01\x00\x00j\x84\x01\x00\x00\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S1_U1_R1.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S2_U1_R1.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S2_U2_R1.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S3_U1_R1.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S1_U1_R2.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S2_U1_R2.fastq.gz\x94\x8cH/redser4/personal/andrew/src/grenepipe/example/samples/S2_U2_R2.fastq.gz\x94G\x7f\xf8\x00\x00\x00\x00\x00\x00et\x94b\x8c\x08builtins\x94\x8c\x05slice\x94\x93\x94K\x00K\x05K\x01\x87\x94R\x94K\x02\x87\x94R\x94\x85\x94]\x94(\x8c\x18pandas.core.indexes.base\x94\x8c\n_new_Index\x94\x93\x94j\x96\x01\x00\x00\x8c\x05Index\x94\x93\x94}\x94(\x8c\x04data\x94jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x05\x85\x94j|\x01\x00\x00\x89]\x94(\x8c\x06sample\x94\x8c\x04unit\x94\x8c\x08platform\x94\x8c\x03fq1\x94\x8c\x03fq2\x94et\x94b\x8c\x04name\x94Nu\x86\x94R\x94j\x98\x01\x00\x00\x8c\x19pandas.core.indexes.multi\x94\x8c\nMultiIndex\x94\x93\x94}\x94(\x8c\x06levels\x94]\x94(j\x98\x01\x00\x00j\x9a\x01\x00\x00}\x94(j\x9c\x01\x00\x00jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x03\x85\x94j|\x01\x00\x00\x89]\x94(\x8c\x02S1\x94\x8c\x02S2\x94\x8c\x02S3\x94et\x94bj\xa8\x01\x00\x00hlu\x86\x94R\x94j\x98\x01\x00\x00j\x9a\x01\x00\x00}\x94(j\x9c\x01\x00\x00jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x02\x85\x94j|\x01\x00\x00\x89]\x94(h\j\x83\x01\x00\x00et\x94bj\xa8\x01\x00\x00hmu\x86\x94R\x94e\x8c\x05codes\x94]\x94(jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x04\x85\x94jy\x01\x00\x00\x8c\x02i1\x94\x89\x88\x87\x94R\x94(K\x03j}\x01\x00\x00NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\x00t\x94b\x89C\x04\x00\x01\x01\x02\x94t\x94bjo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x04\x85\x94j\xce\x01\x00\x00\x89C\x04\x00\x00\x01\x00\x94t\x94be\x8c\tsortorder\x94N\x8c\x05names\x94]\x94(hlhmeu\x86\x94R\x94e\x86\x94R\x94\x8c\x04_typ\x94\x8c\tdataframe\x94\x8c\t_metadata\x94]\x94\x8c\x05attrs\x94}\x94\x8c\x06_flags\x94}\x94\x8c\x17allows_duplicate_labels\x94\x88sub\x8c\x0csample-names\x94]\x94(j\x80\x01\x00\x00j\x81\x01\x00\x00j\x82\x01\x00\x00e\x8c\nunit-names\x94]\x94(j\x83\x01\x00\x00h\e\x8c\x0csample-units\x94]\x94(j\x80\x01\x00\x00h\\x86\x94j\x81\x01\x00\x00h\\x86\x94j\x81\x01\x00\x00j\x83\x01\x00\x00\x86\x94j\x82\x01\x00\x00h\\x86\x94e\x8c\x06fastqc\x94jc\x01\x00\x00)\x81\x94}\x94(jf\x01\x00\x00ji\x01\x00\x00jl\x01\x00\x00jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x04K\x07\x86\x94j|\x01\x00\x00\x89]\x94(j\x80\x01\x00\x00j\x80\x01\x00\x00j\x81\x01\x00\x00j\x81\x01\x00\x00j\x81\x01\x00\x00j\x81\x01\x00\x00j\x82\x01\x00\x00h\h\h\h\j\x83\x01\x00\x00j\x83\x01\x00\x00h\\x8c\x02R1\x94\x8c\x02R2\x94j\xfa\x01\x00\x00j\xfb\x01\x00\x00j\xfa\x01\x00\x00j\xfb\x01\x00\x00j\xfa\x01\x00\x00j\x85\x01\x00\x00j\x89\x01\x00\x00j\x86\x01\x00\x00j\x8a\x01\x00\x00j\x87\x01\x00\x00j\x8b\x01\x00\x00j\x88\x01\x00\x00et\x94bj\x8f\x01\x00\x00K\x00K\x04K\x01\x87\x94R\x94K\x02\x87\x94R\x94\x85\x94]\x94(j\x98\x01\x00\x00j\x9a\x01\x00\x00}\x94(j\x9c\x01\x00\x00jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x04\x85\x94j|\x01\x00\x00\x89]\x94(hlhm\x8c\x02id\x94\x8c\x04file\x94et\x94bj\xa8\x01\x00\x00Nu\x86\x94R\x94j\x98\x01\x00\x00\x8c\x19pandas.core.indexes.range\x94\x8c\nRangeIndex\x94\x93\x94}\x94(j\xa8\x01\x00\x00N\x8c\x05start\x94K\x00\x8c\x04stop\x94K\x07\x8c\x04step\x94K\x01u\x86\x94R\x94e\x86\x94R\x94j\xdf\x01\x00\x00j\xe0\x01\x00\x00j\xe1\x01\x00\x00j\xe2\x01\x00\x00j\xe3\x01\x00\x00}\x94j\xe5\x01\x00\x00}\x94j\xe7\x01\x00\x00\x88sub\x8c\x07contigs\x94\x8c\x12pandas.core.series\x94\x8c\x06Series\x94\x93\x94)\x81\x94}\x94(jf\x01\x00\x00jg\x01\x00\x00\x8c\x12SingleBlockManager\x94\x93\x94)\x81\x94(]\x94j\x98\x01\x00\x00j\x10\x02\x00\x00}\x94(j\xa8\x01\x00\x00Nj\x12\x02\x00\x00K\x00j\x13\x02\x00\x00K\x07j\x14\x02\x00\x00K\x01u\x86\x94R\x94a]\x94jo\x01\x00\x00jr\x01\x00\x00K\x00\x85\x94jt\x01\x00\x00\x87\x94R\x94(K\x01K\x07\x85\x94j|\x01\x00\x00\x89]\x94(h\j\x83\x01\x00\x00\x8c\x013\x94\x8c\x014\x94\x8c\x015\x94\x8c\x0cmitochondria\x94\x8c\x0bchloroplast\x94et\x94ba]\x94j\x98\x01\x00\x00j\x10\x02\x00\x00}\x94(j\xa8\x01\x00\x00Nj\x12\x02\x00\x00K\x00j\x13\x02\x00\x00K\x07j\x14\x02\x00\x00K\x01u\x86\x94R\x94a}\x94\x8c\x060.14.1\x94}\x94(\x8c\x04axes\x94j$\x02\x00\x00\x8c\x06blocks\x94]\x94}\x94(\x8c\x06values\x94j+\x02\x00\x00\x8c\x08mgr_locs\x94j\x8f\x01\x00\x00K\x00K\x07K\x01\x87\x94R\x94uaust\x94bj\xdf\x01\x00\x00\x8c\x06series\x94j\xe1\x01\x00\x00]\x94j\xa8\x01\x00\x00aj\xe3\x01\x00\x00}\x94j\xe5\x01\x00\x00}\x94j\xe7\x01\x00\x00\x88sj\xa8\x01\x00\x00jm\x01\x00\x00\x8c\x06scalar\x94\x93\x94jy\x01\x00\x00\x8c\x02i8\x94\x89\x88\x87\x94R\x94(K\x03\x8c\x01<\x94NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\x00t\x94bC\x08\x00\x00\x00\x00\x00\x00\x00\x00\x94\x86\x94R\x94ubuu\x8c\x04rule\x94\x8c\rtrim_reads_pe\x94\x8c\x0fbench_iteration\x94K\x00\x8c\tscriptdir\x94\x8cMhttps://github.com/snakemake/snakemake-wrappers/raw/0.74.0/bio/trimmomatic/pe\x94ub.'); from snakemake.logging import logger; logger.printshellcmds = False; __real_file = file; file__ = 'https://github.com/snakemake/snakemake-wrappers/raw/0.74.0/bio/trimmomatic/pe/wrapper.py'; AttributeError: Can't get attribute '_unpickle_block' on <module 'pandas._libs.internals' from '/redser4/personal/andrew/src/grenepipe/example/.snakemake/conda/6fd3e3c20e70b66fb92639ab0919e7b2/lib/python3.7/site-packages/pandas/_libs/internals.cpython-37m-x86_64-linux-gnu.so'> [Thu Apr 28 15:56:00 2022] Error in rule trim_reads_pe: jobid: 21 output: trimmed/S2-1.1.fastq.gz, trimmed/S2-1.2.fastq.gz, trimmed/S2-1.1.unpaired.fastq.gz, trimmed/S2-1.2.unpaired.fastq.gz log: logs/trimmomatic/S2-1.log (check log file(s) for error message) conda-env: /redser4/personal/andrew/src/grenepipe/example/.snakemake/conda/6fd3e3c20e70b66fb92639ab0919e7b2

RuleException: CalledProcessError in line 50 of /redser4/personal/andrew/src/grenepipe/rules/trimming-trimmomatic.smk: Command 'source /home/asharo/miniconda3/bin/activate '/redser4/personal/andrew/src/grenepipe/example/.snakemake/conda/6fd3e3c20e70b66fb92639ab0919e7b2'; set -euo pipefail; python /redser4/personal/andrew/src/grenepipe/example/.snakemake/scripts/tmpmpp3gy6n.wrapper.py' returned non-zero exit status 1. File "/redser4/personal/andrew/src/grenepipe/rules/trimming-trimmomatic.smk", line 50, in __rule_trim_reads_pe File "/home/asharo/miniconda3/envs/snakemake2/lib/python3.9/concurrent/futures/thread.py", line 58, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2022-04-28T154328.788125.snakemake.log

As a note, this error appears to occur whether I use trimmomatic or not. Any thoughts on what might be causing this?

[lczech]

Hi @andrewSharo,

yes, that error is most likely due to a mismatch in the version of Pandas on your system and the one used by Snakemake/grenepipe. It is unfortunate that the modern software stack with tools such as conda/mamba (which were invented to avoid these kinds of dependency/version issues...) still suffers from that. Sorry for this. I'm aware of the issue, and currently working on a proper solution.

In the meantime, you can try this: Create a conda environment with the correct Pandas version, and run the pipeline with that environment activated:

mamba create -c conda-forge -c bioconda -n snakemake-5.32.1 snakemake=5.32.1 pandas=1.3.1
conda activate snakemake-5.32.1
<run pipeline as described in the wiki>

I highly recommend mamba here instead of conda to speed up the installation of the environment. This is also better for running snakemake itself, for example

snakemake --conda-frontend mamba [further options as needed]

is what I usually use.

Let me know if that solved your issue (for now). Cheers Lucas

[andrewSharo]

Hi Lucas,

It was great meeting you yesterday at BAPG. Thanks again for your quick reply. Your solution has indeed fixed the error. However, in running the example, I've hit a new error, pasted below. I interpret this to be a permission error. But I'm not sure why this is happening. I am running the example from within the grenepipe directory. Does grenepipe write outside of the directory? Let me know if you have any ideas.

Best, Andrew

Error in rule call_variants: jobid: 104 output: called/S3.mitochondria.g.vcf.gz log: logs/gatk/haplotypecaller/S3.mitochondria.log (check log file(s ) for error message) conda-env: /redser4/personal/andrew/src/grenepipe/example/.snakemake /conda/bac77ba8

Traceback (most recent call last): File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/executors/init.py", line 569, in _callback callback(job) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/scheduler.py", line 533, in _proceed self.get_executor(job).handle_job_success(job) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/executors/init.py", line 583, in handle_job_succ ess super().handle_job_success(job) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/executors/init.py", line 252, in handle_job_succ ess job.postprocess( File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/jobs.py", line 977, in postprocess self.dag.check_and_touch_output( File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/dag.py", line 511, in check_and_touch_output f.touch() File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/io.py", line 650, in touch raise e File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/io.py", line 640, in touch lutime(self.file, times) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/io.py", line 67, in lutime os.utime(f, times, follow_symlinks=False) PermissionError: [Errno 13] Permission denied Removing output files of failed job call_variants since they might be co rrupted: called/S3.mitochondria.g.vcf.gz [Sun May 1 14:56:53 2022] Error in rule call_variants: jobid: 96 output: called/S3.5.g.vcf.gz log: logs/gatk/haplotypecaller/S3.5.log (check log file(s) for error message) conda-env: /redser4/personal/andrew/src/grenepipe/example/.snakemake /conda/bac77ba8

Traceback (most recent call last): File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/executors/init.py", line 569, in _callback callback(job) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/scheduler.py", line 533, in _proceed self.get_executor(job).handle_job_success(job) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/executors/init.py", line 583, in handle_job_succ ess super().handle_job_success(job) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/executors/init.py", line 252, in handle_job_succ ess job.postprocess( File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/jobs.py", line 977, in postprocess self.dag.check_and_touch_output( File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/dag.py", line 511, in check_and_touch_output f.touch() File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/io.py", line 650, in touch raise e File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/io.py", line 640, in touch lutime(self.file, times) File "/home/asharo/miniconda3/envs/snakemake-5.32.1/lib/python3.9/site -packages/snakemake/io.py", line 67, in lutime os.utime(f, times, follow_symlinks=False) PermissionError: [Errno 13] Permission denied Removing output files of failed job call_variants since they might be co rrupted: called/S3.5.g.vcf.gz Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /redser4/personal/andrew/src/grenepipe/example/.snakemake/ log/2022-05-01T143548.761270.snakemake.log

[lczech]

Hi Andrew,

yes, it was indeed great meeting you, and thanks for your positive feedback!

The error that you are seeing now is due to the following: I've marked the results of the SNP caller as write-protected in the pipeline, in order to avoid accidental re-computation. Apparently, somehow snakemake thinks that one of the input files changed, and so it wants to re-compute that file (called/S3.mitochondria.g.vcf.gz). Likely this is due to you trying to run the pipeline before, but with the error that happend with pandas, snakemake aborted with an imcomplete file, still made it write protected, and now cannot overwrite it... I don't know why snakemake does not recognize that the file is incomplete... but okay, that's how it is.

I have had this "problem" in the past as well (which as said in fact was originally meant as a protection mechanism). The simple solution is to run rm called/S3.* to delete all files of that sample, and run again. However, since this is not the first time that this has happened, I'm also considering to simply remove the write-protection from these files in the pipeline. That in turn might cause accidental re-computations in cases where the pipeline succeeded, but will avoid the error you see now when the pipeline failed before.

Thanks for bringing this up, and hope that this fixes it ;-) Lucas

[andrewSharo]

Hi Lucas,

I left out some key information. Before running grenepipe, I deleted the entire directory and re-cloned it from github. So this error occurred on my first run, with a fresh version of grenepipe. Following your directions, I also was not able to find the called directory. It wasn't in the grenepipe base directory or in the example directory. So perhaps this is a slightly different issue?

I will try rm -rf on the entire grenepipe, re-cloning from github, and running again just in case something weird happened. But wanted to update you here first in case you had any other ideas.

Thanks for your help with this. I know how challenging it can be to find all the bugs when you develop a method. I think we are close!

Best, Andrew

[andrewSharo]

Hi Lucas,

I tried removing grenepipe, re-cloning from github, and re-running, but I get the same error.

Best, Andrew

[lczech]

Hi @andrewSharo,

from the email that you send me, it seems that you have solved the issue already - happy to hear :-)

I have also added a conda environment for running snakemake itself, as now documented here, which should solve the first problem you had, and I've removed the write-protection from some intermediate files, which should solve your second problem.

Hence, I'm going to close this issue now. Feel free to re-open or start another should you encounter any further issues! Lucas