Issue Running make_lastz_chains-2.0.6 on Test Data

[ZhaoHang-bio]

Hello,

I encountered an issue when running the test data with the following command: make_chains.py target query test_data/test_reference.fa test_data/test_query.fa --pd test_out -f --chaining_memory 16

The error I received is as follows:

### Nextflow process chain_run finished successfully
An error occurred while executing chain_run: Error! No non-empty files found at /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chain. The failed operation label is: chain_run

Traceback (most recent call last):
  File "/home/zhaohang/soft/make_lastz_chains-2.0.6/modules/step_manager.py", line 70, in execute_steps
    step_result = step_to_function[step](params, project_paths, step_executables)
  File "/home/zhaohang/soft/make_lastz_chains-2.0.6/modules/pipeline_steps.py", line 64, in chain_run_step
    do_chain_run(params, project_paths, executables)
  File "/home/zhaohang/soft/make_lastz_chains-2.0.6/steps_implementations/chain_run_step.py", line 113, in do_chain_run
    has_non_empty_file(project_paths.chain_output_dir, "chain_run")
  File "/home/zhaohang/soft/make_lastz_chains-2.0.6/modules/common.py", line 51, in has_non_empty_file
    raise PipelineFileNotFoundError(err_msg)
modules.error_classes.PipelineFileNotFoundError: Error! No non-empty files found at /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chain. The failed operation label is: chain_run

Any assistance on this would be greatly appreciated.

Thank you!

[kirilenkobm]

Hi @ZhaoHang-bio

Thanks for the bug report.

pls provide the following files: test_out/run.log and test_out/temp_chain_run/chains_jobslist

[ZhaoHang-bio]

The content of run.log is as follows:

# Make Lastz Chains #
Version 2.0.6
Commit: unknown
Branch: unknown

* found run_lastz.py at /home/zhaohang/soft/make_lastz_chains-2.0.6/standalone_scripts/run_lastz.py
* found run_lastz_intermediate_layer.py at /home/zhaohang/soft/make_lastz_chains-2.0.6/standalone_scripts/run_lastz_intermediate_layer.py
* found chain_gap_filler.py at /home/zhaohang/soft/make_lastz_chains-2.0.6/standalone_scripts/chain_gap_filler.py
* found faToTwoBit at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/faToTwoBit
* found twoBitToFa at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/twoBitToFa
* found pslSortAcc at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/pslSortAcc
* found axtChain at /home/zhaohang/anaconda3/envs/TOGA/bin/axtChain
* found axtToPsl at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/axtToPsl
* found chainAntiRepeat at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/chainAntiRepeat
* found chainMergeSort at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/chainMergeSort
* found chainCleaner at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/chainCleaner
* found chainSort at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/chainSort
* found chainScore at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/chainScore
* found lastz at /home/zhaohang/soft/bin/lastz
* found chainFilter at /home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/chainFilter
found chainNet in /home/zhaohang/soft/make_lastz_chains-2.0.6/chain_clean_micro_env
All necessary executables found.
Making chains for test_data/test_reference.fa and test_data/test_query.fa files, saving results to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out
Pipeline started at 2023-09-27 20:34:45.194169
* Setting up genome sequences for target
genomeID: target
input sequence file: test_data/test_reference.fa
is 2bit: False
planned genome dir location: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/target.2bit
Initial fasta file test_data/test_reference.fa saved to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/target.2bit
For target (target) sequence file: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/target.2bit; chrom sizes saved to: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/target.chrom.sizes
* Setting up genome sequences for query
genomeID: query
input sequence file: test_data/test_query.fa
is 2bit: False
planned genome dir location: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/query.2bit
Initial fasta file test_data/test_query.fa saved to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/query.2bit
For query (query) sequence file: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/query.2bit; chrom sizes saved to: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/query.chrom.sizes

### Partition Step ###

# Partitioning for target
Saving partitions and creating 1 buckets for lastz output
In particular, 0 partitions for bigger chromosomes
And 1 buckets for smaller scaffolds
Saving target partitions to: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/target_partitions.txt
# Partitioning for query
Saving partitions and creating 1 buckets for lastz output
In particular, 0 partitions for bigger chromosomes
And 1 buckets for smaller scaffolds
Saving query partitions to: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/query_partitions.txt
Num. target partitions: 0
Num. query partitions: 0
Num. lastz jobs: 0

### Lastz Alignment Step ###

LASTZ: making jobs
LASTZ: saved 1 jobs to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_lastz_run/lastz_joblist.txt
Parallel manager: pushing job nextflow /home/zhaohang/soft/make_lastz_chains-2.0.6/parallelization/execute_joblist.nf --joblist /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_lastz_run/lastz_joblist.txt -c /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_lastz_run/lastz_config.nf

### Nextflow process lastz finished successfully
Found 1 output files from the LASTZ step
Please note that lastz_step.py does not produce output in case LASTZ could not find any alignment

### Concatenating Lastz Results (Cat) Step ###

Concatenating LASTZ output from 1 buckets
* concatenated bucket bucket_ref_bulk_1 to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_concat_lastz_output/concat_0.psl.gz
Concatenated 1 files in total into 1 files

### Build Chains Step ###

Sorting PSL files, saving the results to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/sorted_psl
/home/zhaohang/soft/make_lastz_chains-2.0.6/HL_kent_binaries/pslSortAcc nohead /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/sorted_psl /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_kent /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_concat_lastz_output/concat_0.psl.gz
Bundling psl files with the following arguments:
* input_dir: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/sorted_psl
* chrom_sizes: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/target.chrom.sizes
* output_dir: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/split_psl
* max_bases: 1000000
* warning_only: False
* verbose: False
Saving results to: /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/split_psl
Bundling 2 psl files in total
Written to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/split_psl/bundle.0.psl
DONE. Produced 1 files
PSL bundle sub-step done
Building axtChain joblist for 1 bundled psl files
Saving 1 axtChain jobs to /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chains_joblist
Parallel manager: pushing job nextflow /home/zhaohang/soft/make_lastz_chains-2.0.6/parallelization/execute_joblist.nf --joblist /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chains_joblist -c /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chain_run_config.nf

### Nextflow process chain_run finished successfully
An error occurred while executing chain_run: Error! No non-empty files found at /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chain. The failed operation label is: chain_run

[kirilenkobm]

I have a few more questions:

• does the /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_concat_lastz_output/concat_0.psl.gz contain anything?
• please open /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chains_joblist and execute the command written in this file - does it crash with error or not?

[ZhaoHang-bio]

Thank you for your assistance. When I executed the command in /home/zhaohang/soft/make_lastz_chains-2.0.6/test_out/temp_chain_run/chains_joblist, I found that the axtChain command failed. However, after making the necessary corrections, it is now running successfully. Once again, I appreciate your help.

[kirilenkobm]

@ZhaoHang-bio

Sorry, but I didn't understand - is your problem solved? (if yes - I'd close the issue) Please let me know what corrections did you do in order to make it work. Thanks!

[ZhaoHang-bio]

Yes, I've resolved the issue and it's now running successfully.

The prior failure stemmed from not checking if the UCSC Kent software was functioning properly. Upon running the commands in the chains_joblist file, as per your feedback, I identified the error message as axtChain failed to run, with the error: libssl.so.1.0.0: cannot open shared object file: No such file or directory. By redirecting libssl.so to libssl.so.1.0.0 in the lib path, the program now executes without issues.

Thank you again.

[kirilenkobm]

Nice to hear that it worked for you! Thank you for reporting the issue, I will add the respective note in the project's TODO.