Closed igorpop0v closed 11 months ago
MichaelHiller
commented
1 year ago Can you pls execute /home/popov/make_lastz_chains/kent_binaries/twoBitToFa and check if this is a proper binary that is executable and has user/group/all +r and +x rights?
igorpop0v
commented
1 year ago Dear Prof. Dr. Michael Hiller,
Thank you for your prompt reply.
I executed this command. The result: -bash: /home/popov/make_lastz_chains/kent_binaries/twoBitToFa: Permission denied
Can you please tell if it is possible to correct this issue?
MichaelHiller
commented
1 year ago Pls try chmod 755 /home/popov/make_lastz_chains/kent_binaries/twoBitToFa or better chmod 755 /home/popov/make_lastz_chains/kent_binaries/* assuming that you have binaries in that dir.
igorpop0v
commented
1 year ago Dear Prof. Dr. Michael Hiller,
Thank you very much for your wise advise. Yes, the pipeline worked further after this command: chmod 755 /home/popov/make_lastz_chains/kent_binaries/*
Although, the pipeline still crashes with this error:
Traceback (most recent call last):
File "/home/popov/make_lastz_chains/HL_scripts/parallel_executor.py", line 11, in
Is it possible to correct this error?
MichaelHiller
commented
1 year ago Can you pls talk to your admin whether python and nextflow is correctly installed on your system. The "cannot import name 'Iterable' from 'collections'" is apparently a python version issue, according to https://stackoverflow.com/questions/72032032/importerror-cannot-import-name-iterable-from-collections-in-python
@kirilenkobm Which python version is recommended for the pipeline?
kirilenkobm
commented
1 year ago @MichaelHiller yes, I used Python 3.7 or 3.8 when I was writing this py_nf library.
@igorpop0v sorry for that, could you pls try Python 3.9?
I would need some time to fix this versioning issue.
Thanks.
igorpop0v
commented
1 year ago Dear Dr. Bogdan Kirilenko,
I downgraded Python to 3.9 version. Looks like there is a conflict of this Python version and Nextflow 23.04.1 version:
N E X T F L O W ~ version 23.04.1 Nextflow DSL1 is no longer supported — Update your script to DSL2, or use Nextflow 22.10.x or earlier /home/popov/miniconda3/envs/togaenv/lib/python3.9/site-packages/py_nf/py_nf.py:404: UserWarning: Nextflow pipeline lastz_hg38my_gen failed! Execute function returns 1. warnings.warn(msg) Uncaught exception from user code: Command failed: /home/popov/chains_first_attempt/TEMP_run.lastz/doClusterRun.sh HgAutomate::run('/home/popov/chains_first_attempt/TEMP_run.lastz/doClusterRun.sh') called at /home/popov/make_lastz_chains/doLastzChains/HgRemoteScript.pm line 117 HgRemoteScript::execute('HgRemoteScript=HASH(0x23a3090)') called at /home/popov/make_lastz_chains/doLastzChains/doLastzChain.pl line 423 main::doLastzClusterRun() called at /home/popov/make_lastz_chains/doLastzChains/HgStepManager.pm line 169 HgStepManager::execute('HgStepManager=HASH(0x23a29e8)') called at /home/popov/make_lastz_chains/doLastzChains/doLastzChain.pl line 877 Error!!! Output file /home/popov/chains_first_attempt/hg38.my_gen.allfilled.chain.gz not found! The pipeline crashed. Please contact developers by creating an issue at: https://github.com/hillerlab/make_lastz_chains
Is it possible to downgrade Nextflow to 22.10.x or earlier version?
Thank you for your kind replies. I enjoy using your software and looking forward to using TOGA after creating the chain file.
kirilenkobm
commented
1 year ago @igorpop0v I'm also sorry for that. Nextflow versions above 22.10 no longer support DSL1. I've fixed this for TOGA but haven't managed to do so for this pipeline yet. Downgrading Nextflow should help.
kirilenkobm
commented
11 months ago @igorpop0v An updated and completely rearranged pipeline version will be released today/tomorrow, under the '2.0.0' tag. The nextflow issue will be fixed there (hopefully, new problems will not arise). If you experience any issues with the updated pipeline version, please create a new issue.
ohdongha
commented
4 months ago Sorry for hitchhiking, but one question @kirilenkobm
yes, I used Python 3.7 or 3.8 when I was writing this py_nf library.
The older make_lastz_chains (v. 1.0.0) that used py_nf library was not compatible with Python 3.11.
I noticed that py_nf was removed from `requirements.txt' in the new version (2.0.8). Is the new version tested in Python 3.11?
Thanks! Dong-Ha
Dear Developers,
Than you very much for your software. I tried to make chains file and the following message occurred:
/home/popov/make_lastz_chains/make_chains.py hg38 my_gen /home/popov/Human_genome_38/GRCh38.primary_assembly.genome.ordered.fasta /home/popov/primary.genome.scf.fasta.scaffolds.fa --project_dir /home/popov/chains_first_attempt --executor slurm --executor_partition genetics Project directory: /home/popov/chains_first_attempt Target path: /home/popov/chains_first_attempt/hg38.2bit | chrom sizes: /home/popov/chains_first_attempt/hg38.chrom.sizes Query path: /home/popov/chains_first_attempt/my_gen.2bit | query sizes: /home/popov/chains_first_attempt/my_gen.chrom.sizes Calling: /home/popov/chains_first_attempt/master_script.sh... BIN: /home/popov/make_lastz_chains/doLastzChains which: no RepeatFiller.py in (/home/popov/make_lastz_chains/GenomeAlignmentTools/src:/home/popov/make_lastz_chains/kent_binaries:/home/popov/make_lastz_chains/HL_kent_binaries:/home/popov/make_lastz_chains/HL_scripts:/home/popov/make_lastz_chains/doLastzChains:/home/popov/miniconda3/envs/togaenv/bin:/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.3/bin:/opt/ohpc/pub/compiler/gcc/8.3.0/bin:/opt/ohpc/pub/utils/prun/1.3:/opt/ohpc/pub/utils/autotools/bin:/opt/ohpc/pub/bin:/usr/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/popov/.local/bin:/home/popov/bin) PARAMETERS: clusterRunDir /home/popov/chains_first_attempt chainMinScore 1000 chainLinearGap loose maxNumLastzJobs 6000 numFillJobs 1000 verbose 1 debug 0 /home/popov/chains_first_attempt/DEF looks OK! tDb=hg38 qDb=my_gen seq1dir=/home/popov/chains_first_attempt/hg38.2bit seq2dir=/home/popov/chains_first_attempt/my_gen.2bit Will run RepeatFiller. Will run chainCleaner with parameters: -LRfoldThreshold=2.5 -doPairs -LRfoldThresholdPairs=10 -maxPairDistance=10000 -maxSuspectScore=100000 -minBrokenChainScore=75000 max number of lastz cluster jobs: 6000 The /home/popov/make_lastz_chains/doLastzChains/doLastzChain.pl run will be performed in directory /home/popov/chains_first_attempt. All temp files will be written there. HgStepManager: executing from step 'partition' through step 'cleanChains'.
**** HgStepManager: executing step 'partition' Thu Sep 7 17:58:05 2023. doPartition .... doPartition: seq2MaxLength .... doPartition: seq2MaxLength = 57195878 doPartition: partitionTargetCmd partitionSequence.pl 175000000 0 /home/popov/chains_first_attempt/hg38.2bit /home/popov/chains_first_attempt/hg38.chrom.sizes -xdir xdir.sh -rawDir /home/popov/chains_first_attempt/TEMP_psl 4000 -lstDir tParts > hg38.lst doPartition: partitionQueryCmd partitionSequence.pl 50000000 10000 /home/popov/chains_first_attempt/my_gen.2bit /home/popov/chains_first_attempt/my_gen.chrom.sizes 10000 -lstDir qParts > my_gen.lst content of /home/popov/chains_first_attempt/TEMP_run.lastz/doPartition.bash
chmod a+x /home/popov/chains_first_attempt/TEMP_run.lastz/doPartition.bash
/home/popov/chains_first_attempt/TEMP_run.lastz/doPartition.bash
Can you please tell what could be the reason of this problem and how to solve it?