search

ebi-pf-team / interproscan

Genome-scale protein function classification
Apache License 2.0
292 stars 67 forks source link

test completes successfully but real data crashes . . . #332

Closed benyoung93 closed 1 year ago

benyoung93 commented 1 year ago

Good morning

So I am stumped. As the title says the test data runs successfully and produces all outputs that are expected and there are no error messages. (interproscan.sh -i test_all_appl.fasta -f TSV,XML,GFF3 -dp and interproscan.sh -i test_all_appl.fasta -f tsv)

but now when i run this with my actual protein.fasta file, I am getting the good old problem of it not having the libpcre.so library. For context, I have read all the other issues and in my properties file I have the centos07 from the alt bin provided in the download.

For reference, I have done this through mamba and as I said, the test data all runs properly. Any and all help would be greatly appreciated. 

java.lang.IllegalStateException: Command line failed with exit code: 126
Command: bin/phobius/1.01/phobius.pl /scratch/projects/omics/ofav_genome/funannotate/interproscan_res/temp/n501_20230722_144547283_226t//jobPhobius/000000000001_000000000006.fasta 
Error output from binary:
Phobius ver 1.01
(c) 2004 Lukas Kall, Anders Krogh, Erik Sonnhammer

Could not read provided fasta sequence at bin/phobius/1.01/phobius.pl line 408.

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor89.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.IllegalStateException: Command line failed with exit code: 1
Command: python3 bin/prosite/runprosite.py data/prosite/2022_01/profile_models /scratch/projects/omics/ofav_genome/funannotate/interproscan_res/temp/n501_20230722_144547283_226t//jobPrositeProfiles/000000000001_000000000006.fasta /scratch/projects/omics/ofav_genome/funannotate/interproscan_res/temp/n501_20230722_144547283_226t//jobPrositeProfiles/000000000001_000000000006.raw.out bin/prosite/altbin/pfsearchV3.centos7 -f -o 7 -t 4 
Error output from binary:
bin/prosite/altbin/pfsearchV3.centos7: error while loading shared libraries: libpcre.so: cannot open shared object file: No such file or directory
Error running prosite binary bin/prosite/altbin/pfsearchV3.centos7

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor89.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.IllegalStateException: Command line failed with exit code: 25
Command: bin/prosite/ps_scan.pl -d data/prosite/2022_01/prosite_patterns.dat --pfscan bin/prosite/altbin/pfscanV3.centos7 -b data/prosite/2022_01/evaluator.dat -r -s -o ipro /scratch/projects/omics/ofav_genome/funannotate/interproscan_res/temp/n501_20230722_144547283_226t//jobPrositePatterns/000000000001_000000000006.fasta 
Error output from binary:
bin/prosite/altbin/pfscanV3.centos7: error while loading shared libraries: libpcre.so: cannot open shared object file: No such file or directory
Could not execute bin/prosite/altbin/pfscanV3.centos7 --matrix-only -o4 -L-1 /tmp/ps825721-1.tmp /tmp/ps825721-3.tmp  > /tmp/ps825721-2.tmp: Inappropriate ioctl for device at bin/prosite/ps_scan.pl line 1751.

    at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor89.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1186)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1176)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1073)
    at java.base/java.lang.Thread.run(Thread.java:834)
benyoung93 commented 1 year ago

Okay a small update, I have been running this on my cluster. So when I 

mamba activate interproscan_env
interproscan.sh -i test_all_appl.fasta -f TSV,XML,GFF3 -dp

This successfully runs

BUT

when I submit this as a job *using lsf on my cluster, this is when I get the above message.

Prior to submitting the job mamba activate interproscan_env

then submitting the job 

#BSUB -P coral_omics
#BSUB -q bigmem
#BSUB -J interproscan
#BSUB -n 8
#BSUB -R "rusage[mem=20000]"
#BSUB -e /scratch/projects/omics/ofav_genome/funannotate/interproscan_res/interproscan_test.err
#BSUB -o /scratch/projects/omics/ofav_genome/funannotate/interproscan_res/interproscan_test.out

interproscan.sh -i /nethome/bdy8/mambaforge/envs/interproscan_env/share/InterProScan/test/test_all_appl.fasta \
-f TSV,XML,GFF3 \
-d /scratch/projects/omics/ofav_genome/funannotate/interproscan_res \
-dp

Doing this causes the libpcre.so as stated ebove (with the error message).

I am beyond stumped to why this is happening.

matthiasblum commented 1 year ago

Hi @benyoung93,

You locate the the libpcre.so library is with the following command:

locate libpcre.so

You could then add the directory of the library to your LD_LIBRARY_PATH environment variable, and make sure it's propagated to your LSF job.

Alternatively, you can use InterProScan5 Docker image.