Closed Peter-Zeller closed 1 year ago
Let me know if your issues are resolved now
Hey Buys,
Sorry I missed that email. I reinstalled Single cell multiomics and retried with the standard scChIC pipline, but still get the same tagging error:
Stdout:
Forcing one contig per process
Tagging using multi-processing
Stderr:
NotImplementedError: A blacklist is currently incompatible with --multiprocessing in single contig mode
That really Seems to be a resistant One.
Cheers,
Peter
From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Friday, 19 August 2022 at 15:55 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)
Let me know if your issues are resolved now
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
I think you need to verify that you really updated to the latest version. It should first print "Preparing to blacklist regions from xxx".
Yes you were right. I was still 1 version behind. Sorry about that. Unfortunatly that didn’t solve it yet. The new error is:
Error in rule sort:
jobid: 7
output: processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam, processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam.bai
shell:
samtools sort -T processed/PZ-DRO-bulk-H3K4me2-Th-400/temp_sort -@ 1 processed/PZ-DRO-bulk-H3K4me2-Th-400/unsorted.bam > processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.unfinished.bam && mv processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.unfinished.bam processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam && samtools index processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
cluster_jobid: 11184553
Error executing rule sort on cluster (jobid: 7, external: 11184553, jobscript: /hpc/hub_oudenaarden/Peter/data/EMBL/.snakemake/tmp.2xjti1eh/snakejob.sort.7.sh). For error details see the cluster log and the log files of the involved rule(s).
Submitted job 13 with external jobid '11184588'.
Trying to restart job 19.
Trying to restart job 7.
Select jobs to execute...
I couldn’t find any sort log files though
From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Thursday, 1 September 2022 at 20:37 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)
I think you need to verify that you really updated to the latest version. It should first print "Preparing to blacklist regions from xxx".
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Your disk is most probably full then.
Hey Buys,
I think you were right for the first time I had that error, but even with 1.5TB free it still fails.
Stdout:
Loaded reference from /hpc/hub_oudenaarden/Peter/genemodels/dm6/Drosophila_melanogaster.BDGP6.32.dna.toplevel.fa
Forcing one contig per process
Preparing to blacklist regions from /hpc/hub_oudenaarden/Peter/genemodels/dm6/dm6-blacklist.v2.bed
Sorting
Taking complement
Now subsetting to ./temp_subset_f1679152-0e28-4f13-8f82-eb3670925be5.bam
Proceeding with tagging process
Tagging using multi-processing
Merging final bam files
Stderr:
Traceback (most recent call last):
File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/bin/bamtagmultiome.py", line 1271, in
run_multiome_tagging(args)
File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/bin/bamtagmultiome.py", line 1236, in run_multiome_tagging
job_bed_file=args.jobbed
File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/bin/bamtagmultiome.py", line 422, in tag_multiome_multi_processing
merge_bams(list(tagged_bam_generator), out_bam_path, threads=n_threads)
File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/lib/python3.7/site-packages/singlecellmultiomics/bamProcessing/bamFunctions.py", line 176, in merge_bams
pysam.index(output_path, f'-@ {threads}')
File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/lib/python3.7/site-packages/pysam/utils.py", line 61, in call
save_stdout=kwargs.get("save_stdout", None))
File "pysam/libcutils.pyx", line 323, in pysam.libcutils._pysam_dispatch
OSError: No such file or directory: 'processed/PZ-DRO-bulk-H3K27ac-100/tagged.bam'
It also looks like nothing is really failing, but. It doesn’t find the file. At the same time it fills the folder with numerous tmp files and folders that aren’t deleted anymore.
Marloes will also try to run it, to make sure its not just me.
In case Yake and Alexander don’t keep Maria updated anymore. ChIX was today formally accepted in nature biotech. Congratulations to both of you!
From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Saturday, 3 September 2022 at 09:41 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)
Your disk is most probably full then.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
That is great ! I think the issue is fixed, the cause was the huge amount of tiny contigs in that reference.
Thank you Buys that solved it. I still get errors from the coverage function, but those I am fine ignoring
Thank you for the help!
From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Tuesday, 13 September 2022 at 11:57 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)
That is great ! I think the issue is fixed, the cause was the huge amount of tiny contigs in that reference.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
When running bamtagmultiome.py as part of the Snakemake ChIC workflow it cancels due to incompatibility:
line 330, in tag_multiome_multi_processing raise NotImplementedError('A blacklist is currently incompatible with --multiprocessing in single contig mode') NotImplementedError: A blacklist is currently incompatible with --multiprocessing in single contig mode
single contig mode is not set but get enforced anyway. blacklist usage is only possible with multiprocessing