shilpagarg / DipAsm

MIT License
74 stars 18 forks source link

Fast and accurate chromosome-scale phased assembly #16

Closed shilpagarg closed 3 years ago

shilpagarg commented 3 years ago

Dear users,

Sorry for the late reply. Dipasm is a proof-of-concept tool. Keeping in mind user-friendliness, I recommend the following instructions for fast and accurate chromosome-scale phased assembly using hifi and hi-c:

1) Produce hifiasm r_utg graph. Binary: https://pstools.s3.us-east-2.amazonaws.com/hifiasm 2) Pstools binary: https://pstools.s3.us-east-2.amazonaws.com/pstools_1.

./hifiasm -o pgp1.asm -t60 ../data/pgp1/hifi/*.fastq.gz
awk '/^S/{print ">"$2;print $3}' pgp1.asm.r_utg.gfa > pgp1.asm.r_utg.fa;
./pstools hic_mapping_unitig -t64 pgp1.asm.r_utg.fa <(zcat ../data/PGP1/hic/SRR8310069_1.fastq.gz ../data/PGP1/hic/SRR8310062_1.fastq.gz) <(zcat ../data/PGP1/hic/SRR8310069_2.fastq.gz ../data/PGP1/hic/SRR8310062_2.fastq.gz); ./pstools resolve_haplotypes -t64 hic_name_connection.output pgp1.asm.r_utg.gfa ./; ./pstools hic_mapping_haplo -t64 pred_haplotypes.fa <(zcat ../data/PGP1/hic/SRR8310069_1.fastq.gz ../data/PGP1/hic/SRR8310062_1.fastq.gz) <(zcat ../data/PGP1/hic/SRR8310069_2.fastq.gz ../data/PGP1/hic/SRR8310062_2.fastq.gz) -o scaff_connections.txt; ./pstools haplotype_scaffold -t64 scaff_connections.txt pred_haplotypes.fa ./

Please update data files as per your use case.

In theory, it should work for plants. Please let me know if you have any questions.

Look forward to working with you.

weihongqi commented 3 years ago

Dear Shilpagarg,

Thanks for the instructions. I am following the pstools commands.

Is there a way to estimate the amount of RAM pstools resolve_haplotypes will need?

My "hic_name_connection.output" is around 70G when raw HiC reads were provided. "pstools resolve_haplotypes" ran out of RAM with a 1 TB server.

Should the analysis starts with the set of HiC reads that proven to be "valid tags"? Or the validation of HiC tags were included in the workflow?

Many thanks in advance. Have a nice weekend,

Weihong

shilpagarg commented 3 years ago

<150 Gb for humans with ~30x Hi-C.

weihongqi commented 3 years ago

Dear Shilpagarg,

Many thanks for your prompt response.

For the new analysis, only valid HiC tags after filtering were used for mapping, and the resulting "hic_name_connection.output" is only 7G. But "pstools resolve_haplotypes" still failed when 3 TB RAM was allocated.

The genome is highly heterozygous. Usually the primary assembly needs to be purged aggressively at the end. But each of the haploid genome size is less than 1 Gbp.

The stderr of "pstools resolve_haplotypes": start main start get bubbles rm: missing operand Try 'rm --help' for more information. finish get bubble chain utg... ... (here total 687 lines of utg ids) start get bubbles /var/lib/slurm/slurmd/job3498159/slurm_script: line 8: 1537222 Killed pstools resolve_haplotypes -t32 hic_name_connection.output s60.asm.r_utg.gfa ./

The tmp output "pstools.clean_graph.temp.0.out" is about 1.6 G.

If you have any suggestions on how to further proceed, they will be highly appreciated. Many thanks in advance.

Kind regards,

Weihong

shilpagarg commented 3 years ago

Thank you. If you could send me properties of the genome as well as datasets used at shilpa.garg2k7@gmail.com, I will be happy to help.

shilpagarg commented 3 years ago

@tcb72: Please download binaries and use them as mentioned in the issue. We have tested on several non-human genomes and it seems to work fine. I look forward to helping you with your experiments using pstools (You don't need to install docker etc.).

zhaotao1987 commented 2 years ago

Thanks for the information here, I just wondered would DipAsm work for a triploid species? What's its main advantage comparing with hifiasm?

chunlinxiao commented 6 months ago

is the link https://pstools.s3.us-east-2.amazonaws.com/pstools_1 still working?