search

ebi-pf-team / interproscan

Genome-scale protein function classification
Apache License 2.0
303 stars 67 forks source link

singularity run error #354

Closed smoothyly closed 7 months ago

smoothyly commented 7 months ago

when i run the slurm script

#!/bin/bash
#SBATCH --job-name=uniprot
#SBATCH --partition=A100
#SBATCH --nodes=1
#SBATCH --mail-type=ALL
#SBATCH --mail-user=1041853545@qq.com
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=1
#SBATCH --mem=100gb
#SBATCH --time=14-00:00:00
#SBATCH --output=test.out.%j
#SBATCH --error=test.err.%j

singularity run \
-B /home/liuli/:/data \
-B /backup/liuli/super/interproscan/interproscan-5.67-99.0/data:/opt/interproscan/data \
./interproscan_5.63-95.sif \
-i ./new.fa \
-d outfile \
-f tsv \
-g \
-p \
-c \
-n biocurator \
-x 109069 \
-D database \
-l

it's references to https://interproscan-docs.readthedocs.io/en/latest/HowToUseViaContainer.html and I got a error 

splitfasta.pl -f /data/inputnostar.fasta -s query -o /data/split -n 1000
splitfasta DONE
log4j:WARN No appenders could be found for logger (org.apache.activemq.broker.BrokerService).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
java.lang.IllegalStateException: Command line failed with exit code: 1
Command: bin/hmmer/hmmer3/3.3/hmmsearch -Z 15364 --cut_tc --cpu 1 -o /home/liuli/temp/gpu02_20240403_152218278_b15u//jobNCBIfam/000000000001_000000000117.raw.out data/ncbifam/12.0/ncbifam.hmm /home/liuli/temp/gpu02_20240403_152218278_b15u//jobNCBIfam/000000000001_000000000117.fasta 
Error output from binary:

Error: File existence/permissions problem in trying to open HMM file data/ncbifam/12.0/ncbifam.hmm.
HMM file data/ncbifam/12.0/ncbifam.hmm not found (nor an .h3m binary of it)

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at jdk.proxy2/jdk.proxy2.$Proxy142.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.IllegalStateException: Command line failed with exit code: 1
Command: bin/hmmer/hmmer3/3.1b1/hmmsearch -E 0.01 --acc --cpu 1 -o /home/liuli/temp/gpu02_20240403_152218278_b15u//jobPIRSR/000000000001_000000000117.raw.out --domtblout /home/liuli/temp/gpu02_20240403_152218278_b15u//jobPIRSR/000000000001_000000000117.raw.domtblout.out -A /home/liuli/temp/gpu02_20240403_152218278_b15u//jobPIRSR/000000000001_000000000117.raw.align.out data/pirsr/2021_05/sr_hmm_all /home/liuli/temp/gpu02_20240403_152218278_b15u//jobPIRSR/000000000001_000000000117.fasta 
Error output from binary:

Error: File existence/permissions problem in trying to open HMM file data/pirsr/2021_05/sr_hmm_all.
HMM file data/pirsr/2021_05/sr_hmm_all not found (nor an .h3m binary of it)

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at jdk.proxy2/jdk.proxy2.$Proxy142.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.IllegalStateException: Command line failed with exit code: 1
Command: bin/hmmer/hmmer3/3.3/hmmsearch -Z 15364 --cut_tc --cpu 1 -o /home/liuli/temp/gpu02_20240403_152218278_b15u//jobNCBIfam/000000000001_000000000117.raw.out data/ncbifam/12.0/ncbifam.hmm /home/liuli/temp/gpu02_20240403_152218278_b15u//jobNCBIfam/000000000001_000000000117.fasta 
Error output from binary:

Error: File existence/permissions problem in trying to open HMM file data/ncbifam/12.0/ncbifam.hmm.
HMM file data/ncbifam/12.0/ncbifam.hmm not found (nor an .h3m binary of it)

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at jdk.proxy2/jdk.proxy2.$Proxy142.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.IllegalStateException: Command line failed with exit code: 1
Command: bin/hmmer/hmmer3/3.3/hmmsearch -Z 15364 --cut_tc --cpu 1 -o /home/liuli/temp/gpu02_20240403_152218278_b15u//jobNCBIfam/000000000001_000000000117.raw.out data/ncbifam/12.0/ncbifam.hmm /home/liuli/temp/gpu02_20240403_152218278_b15u//jobNCBIfam/000000000001_000000000117.fasta 
Error output from binary:

Error: File existence/permissions problem in trying to open HMM file data/ncbifam/12.0/ncbifam.hmm.
HMM file data/ncbifam/12.0/ncbifam.hmm not found (nor an .h3m binary of it)

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at jdk.proxy2/jdk.proxy2.$Proxy142.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43)
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:169)
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:413)
InterProScan analysis failed. Exception thrown by StandaloneBlackBoxMaster. Check the log file for details
head: cannot open '/data/outfile/query.0.xml' for reading: No such file or directory
tail: cannot open '/data/outfile/query.0.xml' for reading: No such file or directory
head: cannot open '/data/outfile/query.0.gff3' for reading: No such file or directory
ls: cannot access '/data/outfile/*.gff3': No such file or directory
rm: cannot remove '/data/outfile/query*': No such file or directory
rm: cannot remove 'temp/gpu02_20240403_152218278_b15u/jobSMART/.nfs000000000028023000002824': Device or resource busy
rm: cannot remove 'temp/gpu02_20240403_152218278_b15u/jobMobiDBLite/.nfs00000000001ef77b00002826': Device or resource busy
rm: cannot remove 'temp/gpu02_20240403_152218278_b15u/jobFunFam': Directory not empty
rm: cannot remove 'temp/gpu02_20240403_152218278_b15u/jobCDD': Directory not empty
rm: cannot remove 'temp/gpu02_20240403_152218278_b15u/jobPIRSF': Directory not empty
rm: cannot remove 'temp/gpu02_20240403_152218278_b15u/jobSuperFamily': Directory not empty

so, what can i do for my error? Any help would be appreciated.

smoothyly commented 7 months ago

@laisecf Any help would be appreciated

laisecf commented 7 months ago

Hello! I can't reproduce this issue here, but it seems to be a permission problem (what's happening is that, in the process of mounting the volume, singularity doesn't have permission to access the folders where the files are located). The files it's trying to access (e.g. /backup/liuli/super/interproscan/interproscan-5.67-99.0/data/ncbifam/12.0/ncbifam.hmm) actually exist, right? I found this similar problem (maybe it helps): https://github.com/apptainer/singularity/issues/1833

PS: Based on the name of the sif file, the container appears to have been created for InterProScan 5.63-95.0, but you're using data from InterProScan 5.67-99.0, so the job will probably fail.

Thank you!

smoothyly commented 7 months ago

Thank you for your help！ Sadly, I am not a root user. I'm using the school's server. Actually，I am confused about -B parameter ，My interproscan folder is located at /home/liuli/interproscan-5.63-95.0/data ，and my sequence file is located at /home/liuli/ .However, no matter how I set it always produces an error ： head: cannot open '/data/outfile/query.0.xml' for reading: No such file or directory tail: cannot open '/data/outfile/query.0.xml' for reading: No such file or directory head: cannot open '/data/outfile/query.0.gff3' for reading: No such file or directory rm: cannot remove '/data/outfile/query*': No such file or directory rm: cannot remove 'temp/gpu04_20240411_113001405_jkit/jobMobiDBLite': Directory not empty rm: cannot remove 'temp/gpu04_20240411_102151066_6ryw/jobGene3d/.nfs00000000006138e7000070fe': Device or resource busy rm: cannot remove 'temp/gpu04_20240411_102151066_6ryw/jobGene3d/.nfs00000000006138e6000070ff': Device or resource busy rm: cannot remove 'temp/gpu04_20240411_102151066_6ryw/jobGene3d/.nfs00000000006138e5000070fd': Device or resource busy rm: cannot remove 'temp/gpu04_20240411_112739772_5zpm/jobMobiDBLite/.nfs0000000000616a400000719f': Device or resource busy rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/kvstore/proteinNotInLookupDB': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/kvstore/entryDB': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/kvstore/NucleotideDB': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/kvstore/matchDB': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/kvstore/proteinDB': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobFunFam': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobPrositePatterns': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobGene3d': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobSMART': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobPRINTS': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobMobiDBLite': Directory not empty rm: cannot remove 'temp/gpu04_20240411_104917554_aye9/jobPanther': Directory not empty rm: cannot remove 'temp/gpu04_20240411_112528966_affb/jobMobiDBLite': Directory not empty rm: cannot remove 'output.fasta': No such file or directory I try to search the query.0.xml file ，but it can not be found .

PS: The tutorial https://agbase-docs.readthedocs.io/en/latest/interproscan/using_iprs_cmd.html#running-interproscan-using-docker I referenced shows data from a different version 1712823844330

laisecf commented 7 months ago

Hello! This tutorial documentation is from another organization. Can you try using our image and documentation? https://interproscan-docs.readthedocs.io/en/latest/HowToUseViaContainer.html#using-singularity

Thank you!