Error at Haplotag BAM step

adbeggs commented 1 month ago

Hi

Thank you for this important piece of software. Using fresh pull of singularity image for ClairS-TO, but at the haplotagging stage get the output below.

I've made sure I am not loading a local instance of conda before I run the pipeline on a slurm node, but the image seems to be picking up pysam locally rather than using the instance inside the container - which it can see as it runs fine on other steps.

Command used to run ($A represents sample ID passed from slurm):

singularity exec ~/beggsa-clinicalnanopore/software/clairs-to_latest.sif /opt/bin/run_clairs_to -T "$A".sorted.bam -R ~/beggsa-clinicalnanopore/genomes/grch38/genome.fa -o clairs-to/ -t 32 -p ont_r10_dorado_hac_4khz --conda_prefix /opt/micromamba/envs/clairs-to OS: RedHat Enterprise 8.3

Output:

parsing contig/chromosome: chrEBV ... skip
writeResult ... 2s

[INFO] Haplotag the Tumor BAM
[INFO] RUN THE FOLLOWING COMMAND:
( parallel --joblog /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/logs/phasing_log/parallel_3_haplotag_tumor.log -j 32 /opt/micromamba/envs/clairs-to/bin/whatshap haplotag --output /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/tmp/phasing_output/phased_bam_output/tumor_{1}.bam --reference /rds/homes/b/beggsa/beggsa-clinicalnanopore/genomes/grch38/genome.fa --regions {1}  --ignore-read-groups /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/tmp/phasing_output/phased_vcf_output/tumor_phased_{1}.vcf.gz /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/S475276.sorted.bam :::: /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/tmp/CONTIGS ) 2>&1 | tee /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/logs/phasing_log/3_tumor_haplotag.log && parallel -j 32 samtools index  -@32 /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/tmp/phasing_output/phased_bam_output/tumor_{1}.bam :::: /rds/projects/b/beggsa-sarcomaaccelerator/LongRead/LA95_Sarcoma_03-05-24/S475276/20240503_1646_3A_PAW55915_ceedab86/clairs-to/tmp/CONTIGS

Traceback (most recent call last):
  File "/opt/micromamba/envs/clairs-to/bin/whatshap", line 7, in <module>
    from whatshap.__main__ import main
  File "/opt/micromamba/envs/clairs-to/lib/python3.9/site-packages/whatshap/__main__.py", line 7, in <module>
    import whatshap.cli as cli_package
  File "/opt/micromamba/envs/clairs-to/lib/python3.9/site-packages/whatshap/cli/__init__.py", line 5, in <module>
    from whatshap.bam import (
  File "/opt/micromamba/envs/clairs-to/lib/python3.9/site-packages/whatshap/bam.py", line 6, in <module>
    import pysam
  File "/rds/homes/b/beggsa/.local/lib/python3.9/site-packages/pysam/__init__.py", line 4, in <module>
    from pysam.libchtslib import *
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.27' not found (required by /rds/homes/b/beggsa/.local/lib/python3.9/site-packages/pysam/../pysam.libs/libssh-f4fb36c6.so.4.8.7)
Traceback (most recent call last):
  File "/opt/micromamba/envs/clairs-to/bin/whatshap", line 7, in <module>
    from whatshap.__main__ import main
  File "/opt/micromamba/envs/clairs-to/lib/python3.9/site-packages/whatshap/__main__.py", line 7, in <module>
    import whatshap.cli as cli_package
  File "/opt/micromamba/envs/clairs-to/lib/python3.9/site-packages/whatshap/cli/__init__.py", line 5, in <module>
    from whatshap.bam import (
  File "/opt/micromamba/envs/clairs-to/lib/python3.9/site-packages/whatshap/bam.py", line 6, in <module>
    import pysam
  File "/rds/homes/b/beggsa/.local/lib/python3.9/site-packages/pysam/__init__.py", line 4, in <module>
    from pysam.libchtslib import *
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.27' not found (required by /rds/homes/b/beggsa/.local/lib/python3.9/site-packages/pysam/../pysam.libs/libssh-f4fb36c6.so.4.8.7)

adbeggs commented 1 month ago

run_clairs_to.log Full log attached here.

JasonCLEI commented 1 month ago

Hi, @adbeggs,

Thanks a lot for your interest. In the latest release of ClairS-TO, we replaced default WhatsHap haplotagging with LongPhase haplotagging to speed up the read haplotagging process. You are suggested to re-pull the image and rerun the program with your ONT data with the latest release of ClairS-TO. If the issue still exists, could you please provide your complete running log of run_clairs_to.log file with me, and I will check the issue for you further. In addition, it seems that there is an incompatibility issue between the pysam dependency (which can be chosen to either use bundled htslib or external) and your RedHat system when executing WhatsHap haplotagging based on the original running log.

Lei

aquaskyline commented 1 month ago

@adbeggs Please try the latest version to see if the pysam problem still exists. If yes, we will try setting up RedHat Enterprise 8.3 on our side to figure out a fix.

adbeggs commented 1 month ago

Hi both

Trying now – I have repulled the image, and it is actually RedHat Enterprise 8.8 apologies, but I will let you know what happens.

BW

Andrew

From: Ruibang Luo @.> Sent: Monday, May 20, 2024 11:21 AM To: HKU-BAL/ClairS-TO @.> Cc: Andrew Beggs (Cancer and Genomic Sciences) @.>; Mention @.> Subject: Re: [HKU-BAL/ClairS-TO] Error at Haplotag BAM step (Issue #10)

CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.

@adbeggshttps://github.com/adbeggs Please try the latest version to see if the pysam problem still exists. If yes, we will try setting up RedHat Enterprise 8.3 on our side to figure out a fix.

— Reply to this email directly, view it on GitHubhttps://github.com/HKU-BAL/ClairS-TO/issues/10#issuecomment-2120144809, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC7KTDA7V3BFL3H4URKDNKTZDHFBLAVCNFSM6AAAAABH6F6HOSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRQGE2DIOBQHE. You are receiving this because you were mentioned.Message ID: @.***>

adbeggs commented 1 month ago

Hi both

Seems to be working now with the latest update – at least it’s made it past the stage it finished at last time

BW

Andrew

From: Ruibang Luo @.> Sent: Monday, May 20, 2024 11:21 AM To: HKU-BAL/ClairS-TO @.> Cc: Andrew Beggs (Cancer and Genomic Sciences) @.>; Mention @.> Subject: Re: [HKU-BAL/ClairS-TO] Error at Haplotag BAM step (Issue #10)

CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.

@adbeggshttps://github.com/adbeggs Please try the latest version to see if the pysam problem still exists. If yes, we will try setting up RedHat Enterprise 8.3 on our side to figure out a fix.

— Reply to this email directly, view it on GitHubhttps://github.com/HKU-BAL/ClairS-TO/issues/10#issuecomment-2120144809, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC7KTDA7V3BFL3H4URKDNKTZDHFBLAVCNFSM6AAAAABH6F6HOSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRQGE2DIOBQHE. You are receiving this because you were mentioned.Message ID: @.***>

HKU-BAL / ClairS-TO

Error at Haplotag BAM step #10