bamtagmultiome cannot be run with blacklist

[Peter-Zeller]

When running bamtagmultiome.py as part of the Snakemake ChIC workflow it cancels due to incompatibility:

line 330, in tag_multiome_multi_processing raise NotImplementedError('A blacklist is currently incompatible with --multiprocessing in single contig mode') NotImplementedError: A blacklist is currently incompatible with --multiprocessing in single contig mode

single contig mode is not set but get enforced anyway. blacklist usage is only possible with multiprocessing

[BuysDB]

Let me know if your issues are resolved now

[Peter-Zeller]

Hey Buys,

Sorry I missed that email. I reinstalled Single cell multiomics and retried with the standard scChIC pipline, but still get the same tagging error:

Stdout:

Forcing one contig per process

Tagging using multi-processing

Stderr:

NotImplementedError: A blacklist is currently incompatible with --multiprocessing in single contig mode

That really Seems to be a resistant One.

Cheers,

Peter

From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Friday, 19 August 2022 at 15:55 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)

Let me know if your issues are resolved now

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[BuysDB]

I think you need to verify that you really updated to the latest version. It should first print "Preparing to blacklist regions from xxx".

[Peter-Zeller]

Yes you were right. I was still 1 version behind. Sorry about that. Unfortunatly that didn’t solve it yet. The new error is:

Error in rule sort:

jobid: 7

output: processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam, processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam.bai

shell:

    samtools sort -T processed/PZ-DRO-bulk-H3K4me2-Th-400/temp_sort -@ 1 processed/PZ-DRO-bulk-H3K4me2-Th-400/unsorted.bam > processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.unfinished.bam && mv processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.unfinished.bam processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam && samtools index processed/PZ-DRO-bulk-H3K4me2-Th-400/sorted.bam

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

cluster_jobid: 11184553

Error executing rule sort on cluster (jobid: 7, external: 11184553, jobscript: /hpc/hub_oudenaarden/Peter/data/EMBL/.snakemake/tmp.2xjti1eh/snakejob.sort.7.sh). For error details see the cluster log and the log files of the involved rule(s).

Submitted job 13 with external jobid '11184588'.

Trying to restart job 19.

Trying to restart job 7.

Select jobs to execute...

I couldn’t find any sort log files though

From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Thursday, 1 September 2022 at 20:37 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)

I think you need to verify that you really updated to the latest version. It should first print "Preparing to blacklist regions from xxx".

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[BuysDB]

Your disk is most probably full then.

[Peter-Zeller]

Hey Buys,

I think you were right for the first time I had that error, but even with 1.5TB free it still fails.

Stdout:

Loaded reference from /hpc/hub_oudenaarden/Peter/genemodels/dm6/Drosophila_melanogaster.BDGP6.32.dna.toplevel.fa

Forcing one contig per process

Preparing to blacklist regions from /hpc/hub_oudenaarden/Peter/genemodels/dm6/dm6-blacklist.v2.bed

    Sorting

    Taking complement

    Now subsetting to ./temp_subset_f1679152-0e28-4f13-8f82-eb3670925be5.bam

Proceeding with tagging process

Tagging using multi-processing

Merging final bam files

Stderr:

Traceback (most recent call last):

File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/bin/bamtagmultiome.py", line 1271, in

run_multiome_tagging(args)

File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/bin/bamtagmultiome.py", line 1236, in run_multiome_tagging

job_bed_file=args.jobbed

File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/bin/bamtagmultiome.py", line 422, in tag_multiome_multi_processing

merge_bams(list(tagged_bam_generator), out_bam_path, threads=n_threads)

File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/lib/python3.7/site-packages/singlecellmultiomics/bamProcessing/bamFunctions.py", line 176, in merge_bams

pysam.index(output_path, f'-@ {threads}')

File "/hpc/hub_oudenaarden/Peter/bin/miniconda3/envs/condaP2/lib/python3.7/site-packages/pysam/utils.py", line 61, in call

save_stdout=kwargs.get("save_stdout", None))

File "pysam/libcutils.pyx", line 323, in pysam.libcutils._pysam_dispatch

OSError: No such file or directory: 'processed/PZ-DRO-bulk-H3K27ac-100/tagged.bam'

It also looks like nothing is really failing, but. It doesn’t find the file. At the same time it fills the folder with numerous tmp files and folders that aren’t deleted anymore.

Marloes will also try to run it, to make sure its not just me.

In case Yake and Alexander don’t keep Maria updated anymore. ChIX was today formally accepted in nature biotech. Congratulations to both of you!

From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Saturday, 3 September 2022 at 09:41 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)

Your disk is most probably full then.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[BuysDB]

That is great ! I think the issue is fixed, the cause was the huge amount of tiny contigs in that reference.

[Peter-Zeller]

Thank you Buys that solved it. I still get errors from the coverage function, but those I am fine ignoring

Thank you for the help!

From: Buys de Barbanson @.> Reply-To: BuysDB/SingleCellMultiOmics @.> Date: Tuesday, 13 September 2022 at 11:57 To: BuysDB/SingleCellMultiOmics @.> Cc: Peter-Zeller @.>, Author @.***> Subject: Re: [BuysDB/SingleCellMultiOmics] bamtagmultiome cannot be run with blacklist (Issue #255)

That is great ! I think the issue is fixed, the cause was the huge amount of tiny contigs in that reference.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>