Error executing process > 'THETA_ESTIMATE (1)'

[azk001]

I have submitted this command:

!/bin/bash

source /opt/asn/etc/asn-bash-profiles-special/modules.sh module load samtools module load freebayes/1.0.2 module load bcftools/1.13

for file in /rhometa/bam/*.bam; do tag=${file%.bam};

nextflow run theta_est.nf --bam "$file" --fa reference.fa --output_dir ThetaEst_bam; done

I got this error

executor > local (5) [5b/e5e155] process > FILENAME_PREFIX (1) [100%] 1 of 1 ✔ [ce/216bb2] process > FILTER_BAM (1) [100%] 1 of 1 ✔ [67/678fdf] process > SORT_BAM (1) [100%] 1 of 1 ✔ [c9/fec503] process > FREEBAYES (1) [100%] 1 of 1 ✔ [f4/a9f44d] process > THETA_ESTIMATE (1) [100%] 1 of 1, failed: 1 ✘ Error executing process > 'THETA_ESTIMATE (1)'

Caused by: Process THETA_ESTIMATE (1) terminated with an error exit status (1)

Command executed:

samtools mpileup Aligned_sorted.bam > Aligned_sorted.pileup genome_size=$(samtools view -H Aligned_sorted.bam | grep "@SQ" | awk '{ print $3 }' | cut -c 4-)

theta_est.py $genome_size Aligned_sorted.pileup freebayes_filt.vcf

Command exit status: 1

Command output: (empty)

Command error: /apps/dmc/apps/lmod_rhel/lmod/lmod/init/bash: line 16: lmod_vx: unbound variable [mpileup] 1 samples in 1 input files [E::hts_open_format] Failed to open file "127760" : No such file or directory Traceback (most recent call last): File "/rhometa/bin/theta_est.py", line 92, in num_variant_positions = len(get_var_pos_from_vcf(vcf)) File "/rhometa/bin/theta_est.py", line 15, in get_var_pos_from_vcf f = pysam.VariantFile(vcf_file) File "pysam/libcbcf.pyx", line 4054, in pysam.libcbcf.VariantFile.init__ File "pysam/libcbcf.pyx", line 4279, in pysam.libcbcf.VariantFile.open FileNotFoundError: [Errno 2] could not open variant file b'127760': No such file or directory

Work dir: ./rhometa/work/f4/a9f44d49eb7c781b0d3f697beae788

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

N E X T F L O W ~ version 22.04.5 Launching theta_est.nf [shrivelled_koch] DSL2 - revision: 0511ebd9b4 [- ] process > FILENAME_PREFIX - [- ] process > FILTER_BAM - [- ] process > SORT_BAM - [- ] process > FREEBAYES - [- ] process > THETA_ESTIMATE -

executor > local (1) [75/92a334] process > FILENAME_PREFIX (1) [ 0%] 0 of 1 [- ] process > FILTER_BAM - [- ] process > SORT_BAM - [- ] process > FREEBAYES - [- ] process > THETA_ESTIMATE -

executor > local (2) [75/92a334] process > FILENAME_PREFIX (1) [100%] 1 of 1 ✔ [1b/06b61c] process > FILTER_BAM (1) [ 0%] 0 of 1 [- ] process > SORT_BAM - [- ] process > FREEBAYES - [- ] process > THETA_ESTIMATE -

executor > local (2) [75/92a334] process > FILENAME_PREFIX (1) [100%] 1 of 1 ✔ [1b/06b61c] process > FILTER_BAM (1) [100%] 1 of 1 ✔ [- ] process > SORT_BAM [ 0%] 0 of 1 [- ] process > FREEBAYES - [- ] process > THETA_ESTIMATE -

executor > local (3) [75/92a334] process > FILENAME_PREFIX (1) [100%] 1 of 1 ✔ [1b/06b61c] process > FILTER_BAM (1) [100%] 1 of 1 ✔ [5c/3d2547] process > SORT_BAM (1) [ 0%] 0 of 1 [- ] process > FREEBAYES - [- ] process > THETA_ESTIMATE -

executor > local (4) [75/92a334] process > FILENAME_PREFIX (1) [100%] 1 of 1 ✔ [1b/06b61c] process > FILTER_BAM (1) [100%] 1 of 1 ✔ [5c/3d2547] process > SORT_BAM (1) [100%] 1 of 1 ✔ [d7/01d23c] process > FREEBAYES (1) [ 0%] 0 of 1 [- ] process > THETA_ESTIMATE -

Any insights on troubleshooting this would be helpful.

Thank you.

[sid-krish]

There was an issue in theta_est.nf where it was not supporting bams aligned to a multi-entry reference. This is fixed in the latest release v1.0.2 Please give this version a try and see if it solves the problem.

[azk001]

Thank you for the update. The new version is working till now.

[azk001]

Hi, The new version worked well for the theta_estimation and making look-up tables, however, it didn't work for the rho_estimation. We had an old version so we tried rho estimation with an old version and it worked.

The error got from the new version with rho estimation:

executor > local (8) [ad/4a24a9] process > FILENAME_PREFIX (1) [100%] 1 of 1 ✔ [19/e4aa2c] process > FILTER_BAM (1) [100%] 1 of 1 ✔ [f3/e0c5f4] process > SORT_BAM (1) [100%] 1 of 1 ✔ [b1/cd5408] process > MAKE_PILEUP (1) [100%] 1 of 1 ✔ [99/341035] process > SUBSAMPLE (1) [100%] 1 of 1 ✔ [73/d62335] process > FREEBAYES (1) [100%] 1 of 1 ✔ [3b/aa7102] process > PAIRWISE_TABLE_PAIRED_END (1) [100%] 1 of 1 ✔ [52/f53543] process > RHO_ESTIMATE (1) [100%] 1 of 1, failed: 1 ✘ [- ] process > RESULTS_PLOT - Error executing process > 'RHO_ESTIMATE (1)'

Caused by: Process RHO_ESTIMATE (1) terminated with an error exit status (134)

Command executed:

main_weighted.py 1000 3,85 101,100 pairwise_table.pkl 4

Command exit status: 134

Command output: (empty)

Command error: /apps/dmc/apps/lmod_rhel/lmod/lmod/init/bash: line 16: __lmod_vx: unbound variable LLVM ERROR: Symbol not found: __svml_log4_ha .command.sh: line 2: 113082 Aborted (core dumped) main_weighted.py 1000 3,85 101,100 pairwise_table.pkl 4

Work dir: /rhometa_v2/rhometa/work/52/f535439f954a5e3ec3146cdbc10e1a

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .comman d.sh

Thank you.

[sid-krish]

Hi @azk001, there have only been minor changes to rho_est.nf mostly around organising the output files and reducing memory use in the early subsample process. I think the issue might be due to another factor?

Looking at the command error LLVM ERROR: Symbol not found: __svml_log4_ha, it's possible the dependencies might not have been installed properly? Perhaps using docker/singularity with the image provided might solve the issue. If using conda deleting the work directory will cause nextflow to reinstall the dependencies again.

Let me know how you go. If I can get a copy of the files used for testing, I can also test it on my side.

[azk001]

Thank you for the suggestion to resolve the issue. It helped a lot. The issue with the new one was solved in the older version. Thus I opted that way and got the output. I think it is fine for now. Thank you again!