search

ebi-pf-team / interproscan

Genome-scale protein function classification
Apache License 2.0
300 stars 67 forks source link

Error: Table not found?? #180

Closed srobb1 closed 3 years ago

srobb1 commented 3 years ago

Hello,

I am getting this same error #178 . I split my FASTA into 20 parts and am running consecutively. Not all the parts gave me this error. I am rerunning the parts that did error out and they are giving me the error again. Rerunning is not solving my issue. Any other suggestions?

Thank you, Sofia

30/01/2021 14:56:23:649 Welcome to InterProScan-5.48-83.0
30/01/2021 14:56:23:653 Running InterProScan v5 in STANDALONE mode... on Linux
30/01/2021 14:56:50:329 Loading file /home/smr/project/sanchez/ejr23_trinity_ptychodera_flava/aa/split/Trinity.fasta.transdecoder.clean.pep_part_1.fasta
30/01/2021 14:56:50:332 Running the following analyses:
[CDD-3.18,Coils-2.2.1,Gene3D-4.2.0,MobiDBLite-2.0,PANTHER-15.0,Pfam-33.1,PIRSF-3.10,PRINTS-42.0,ProSitePatterns-2019_11,ProSiteProfiles-2019_11,SFLD-4,SignalP_EUK-4.1,SMART-7.1,SUPERFAMILY-1.75,TIGRFAM-15.0,TMHMM-2.0c]
Pre-calculated match lookup service DISABLED.  Please wait for match calculations to complete...
30/01/2021 15:15:09:636 25% completed
30/01/2021 15:33:15:464 50% completed
30/01/2021 15:39:42:262 75% completed
30/01/2021 15:47:43:033 90% completed
2021-01-30 15:58:32,651 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.RuntimeException: Could not open table 1184
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:99) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:80) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:75) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.Level.get(Level.java:131) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.Version.get(Version.java:172) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577) ~[leveldb-0.12.jar:0.12]
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.get(LevelDBStore.java:109) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.getProtein(ProteinDAOImpl.java:150) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor112.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.getProtein(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:279) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.io.FileNotFoundException: /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001184.sst (No such file or directory)
    at java.io.FileInputStream.open0(Native Method) ~[?:?]
    at java.io.FileInputStream.open(FileInputStream.java:219) ~[?:?]
    at java.io.FileInputStream.<init>(FileInputStream.java:157) ~[?:?]
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:126) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:115) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:68) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:63) ~[leveldb-0.12.jar:0.12]
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache.get(LocalCache.java:3953) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4960) ~[guava-27.1-jre.jar:?]
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:92) ~[leveldb-0.12.jar:0.12]
    ... 44 more
2021-01-30 15:58:32,668 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:32,670 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.RuntimeException: Could not open table 1184
2021-01-30 15:58:32,670 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-01-30 15:58:32,792 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:25] FATAL - Analysis step 4 : Prepare proteins for output of the completed analysis for proteins 4001 to 6000 has failed irretrievably.  Available StackTraces follow.
2021-01-30 15:58:32,793 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:42] FATAL - The JVM will now exit with a non-zero exit status.
2021-01-30 15:58:32,793 [main] [uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster:332] ERROR - Exception thrown by StandaloneBlackBoxMaster:
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43) ~[interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:157) [interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:475) [interproscan-5.jar:?]
2021-01-30 15:58:33,355 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001242.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:345) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-01-30 15:58:33,356 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:33,356 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001242.log
2021-01-30 15:58:33,356 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-01-30 15:58:33,372 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001243.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:345) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-01-30 15:58:33,372 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:33,373 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001243.log
2021-01-30 15:58:33,373 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-01-30 15:58:33,386 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001244.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:345) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

▽
30/01/2021 14:56:23:649 Welcome to InterProScan-5.48-83.0
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-01-30 15:58:33,387 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:33,387 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001244.log
2021-01-30 15:58:33,387 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
15:58:33.401 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
15:58:33.415 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$ vi out.Trinity.fasta.transdecoder.clean.pep_part_1.fasta
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$
smr@ginkgo ~/project/sanchez/ejr23_trinity_ptychodera_flava/aa$ cat out.Trinity.fasta.transdecoder.clean.pep_part_1.fasta
30/01/2021 14:56:23:649 Welcome to InterProScan-5.48-83.0
30/01/2021 14:56:23:653 Running InterProScan v5 in STANDALONE mode... on Linux
30/01/2021 14:56:50:329 Loading file /home/smr/project/sanchez/ejr23_trinity_ptychodera_flava/aa/split/Trinity.fasta.transdecoder.clean.pep_part_1.fasta
30/01/2021 14:56:50:332 Running the following analyses:
[CDD-3.18,Coils-2.2.1,Gene3D-4.2.0,MobiDBLite-2.0,PANTHER-15.0,Pfam-33.1,PIRSF-3.10,PRINTS-42.0,ProSitePatterns-2019_11,ProSiteProfiles-2019_11,SFLD-4,SignalP_EUK-4.1,SMART-7.1,SUPERFAMILY-1.75,TIGRFAM-15.0,TMHMM-2.0c]
Pre-calculated match lookup service DISABLED.  Please wait for match calculations to complete...
30/01/2021 15:15:09:636 25% completed
30/01/2021 15:33:15:464 50% completed
30/01/2021 15:39:42:262 75% completed
30/01/2021 15:47:43:033 90% completed
2021-01-30 15:58:32,651 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.RuntimeException: Could not open table 1184
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:99) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:80) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:75) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.Level.get(Level.java:131) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.Version.get(Version.java:172) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577) ~[leveldb-0.12.jar:0.12]
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.get(LevelDBStore.java:109) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.getProtein(ProteinDAOImpl.java:150) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor112.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.getProtein(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:279) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.io.FileNotFoundException: /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001184.sst (No such file or directory)
    at java.io.FileInputStream.open0(Native Method) ~[?:?]
    at java.io.FileInputStream.open(FileInputStream.java:219) ~[?:?]
    at java.io.FileInputStream.<init>(FileInputStream.java:157) ~[?:?]
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:126) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:115) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:68) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:63) ~[leveldb-0.12.jar:0.12]
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache.get(LocalCache.java:3953) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4960) ~[guava-27.1-jre.jar:?]
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:92) ~[leveldb-0.12.jar:0.12]
    ... 44 more
2021-01-30 15:58:32,668 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:32,670 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.RuntimeException: Could not open table 1184
2021-01-30 15:58:32,670 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-01-30 15:58:32,792 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:25] FATAL - Analysis step 4 : Prepare proteins for output of the completed analysis for proteins 4001 to 6000 has failed irretrievably.  Available StackTraces follow.
2021-01-30 15:58:32,793 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:42] FATAL - The JVM will now exit with a non-zero exit status.
2021-01-30 15:58:32,793 [main] [uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster:332] ERROR - Exception thrown by StandaloneBlackBoxMaster:
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43) ~[interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:157) [interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:475) [interproscan-5.jar:?]
2021-01-30 15:58:33,355 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001242.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:345) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-01-30 15:58:33,356 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:33,356 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001242.log
2021-01-30 15:58:33,356 [amqEmbeddedWorkerJmsContainer-42] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-01-30 15:58:33,372 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001243.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:345) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-01-30 15:58:33,372 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:33,373 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001243.log
2021-01-30 15:58:33,373 [amqEmbeddedWorkerJmsContainer-26] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-01-30 15:58:33,386 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001244.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:345) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-01-30 15:58:33,387 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-01-30 15:58:33,387 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210130_145628472_f7u8/kvstore/proteinDB/001244.log
2021-01-30 15:58:33,387 [amqEmbeddedWorkerJmsContainer-43] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
15:58:33.401 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
15:58:33.415 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
gsn7 commented 3 years ago

hi Sofia, the primary error is occurring at the point where InterProScan is preparing and writing output results to disk and at this time it is I/O intensive. It seems the process is unable to recover from some file system problem. I have developed a possible fix, and I would ask you to test it out on the set of sequences that fail. Let us have the log afterwards, so we can check if the fix is working. the new interproscan is at https://www.ebi.ac.uk/~nuka/files/interproscan/. It uses the same InterPro data as InterProScan-5.48-83.0

srobb1 commented 3 years ago

Hello @gsn7,

Thank you very much for your help. I download the version you posted and ran it. Here is what I get back 

02/02/2021 12:57:11:579 Welcome to InterProScan-5.50RC1-83.0
02/02/2021 12:57:11:583 Running InterProScan v5 in STANDALONE mode... on Linux
02/02/2021 12:57:16:024 RunID: ginkgo.sgc.loc_20210202_125715908_b6lo
02/02/2021 12:57:33:935 Loading file /home/smr/project/sanchez/ejr23_trinity_ptychodera_flava/aa/split/Trinity.fasta.transdecoder.clean.pep_part_1.fasta
02/02/2021 12:57:33:939 Running the following analyses:
[CDD-3.18,Coils-2.2.1,Gene3D-4.2.0,MobiDBLite-2.0,PANTHER-15.0,Pfam-33.1,PIRSF-3.10,PRINTS-42.0,ProSitePatterns-2019_11,ProSiteProfiles-2019_11,SFLD-4,SignalP_EUK-4.1,SMART-7.1,SUPERFAMILY-1.75,TIGRFAM-15.0,TMHMM-2.0c]
Pre-calculated match lookup service DISABLED.  Please wait for match calculations to complete...
02/02/2021 13:15:48:039 25% completed
02/02/2021 13:39:22:106 50% completed
02/02/2021 13:49:38:420 75% completed
02/02/2021 14:02:04:719 90% completed
02/02/2021 14:19:18:868 Info: [7001_7073] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:19:18:871 Info: [7001_7073] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:38:232 Info: [6001_7000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:38:232 Info: [6001_7000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:40:377 Info: [3001_4000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:40:377 Info: [3001_4000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:42:086 Info: [5001_6000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:42:087 Info: [5001_6000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:42:128 Info: [2001_3000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:42:128 Info: [2001_3000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:45:809 Info: [1_1000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:45:809 Info: [1_1000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:47:411 Info: [4001_5000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
02/02/2021 14:20:47:411 Info: [4001_5000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
2021-02-02 14:20:47,426 [amqEmbeddedWorkerJmsContainer-18] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:755) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:533) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:304) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-02-02 14:20:47,441 [amqEmbeddedWorkerJmsContainer-18] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-02-02 14:20:47,443 [amqEmbeddedWorkerJmsContainer-18] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
2021-02-02 14:20:47,443 [amqEmbeddedWorkerJmsContainer-18] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-02-02 14:20:47,604 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:25] FATAL - Analysis step 10 : Prepare proteins for output of the completed analysis for proteins 4001 to 5000 has failed irretrievably.  Available StackTraces follow.
2021-02-02 14:20:47,605 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:42] FATAL - The JVM will now exit with a non-zero exit status.
2021-02-02 14:20:47,605 [main] [uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster:341] ERROR - Exception thrown by StandaloneBlackBoxMaster:
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43) ~[interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:166) [interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:475) [interproscan-5.jar:?]
2021-02-02 14:20:48,363 [amqEmbeddedWorkerJmsContainer-3] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210202_125715908_b6lo/kvstore/proteinDB/002540.log
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.put(LevelDBStore.java:94) ~[interproscan-persistence-5.50RC1-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.persist(ProteinDAOImpl.java:140) ~[interproscan-persistence-5.50RC1-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:48) ~[interproscan-persistence-5.50RC1-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.GenericKVDAOImpl.persist(GenericKVDAOImpl.java:14) ~[interproscan-persistence-5.50RC1-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.persist(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:431) ~[interproscan-management-5.50RC1-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:89) ~[interproscan-management-5.50RC1-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-02-02 14:20:48,364 [amqEmbeddedWorkerJmsContainer-3] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-02-02 14:20:48,365 [amqEmbeddedWorkerJmsContainer-3] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.IllegalStateException: Problem inserting data into the DBStore java.lang.RuntimeException: Unable to open new log file /scratch/smr/ginkgo.sgc.loc_20210202_125715908_b6lo/kvstore/proteinDB/002540.log
2021-02-02 14:20:48,365 [amqEmbeddedWorkerJmsContainer-3] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
14:20:48.383 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
gsn7 commented 3 years ago

the main error is about failing to create a temporary file. Its still appearing at the point when InterProScan is about to write out results. Can you try a different temp directory other than /scratch or /tmp for the option --tempdir? e.g., your home directory or if on a cluster use the file system there. Also temporarily change the following property to false in the file interproscan.properties: delete.temporary.directory.on.completion=false this would allow to verify later if accessing the temporary files is actually the problem.

JonEilers commented 3 years ago

I think I am having a similar issue. 

./interproscan.sh \
> -cpu 75 \
> -d /home/jon/Working_Files/interproscan/platanus-allee_illumina_step10.renamed.masked_repeatmodeler-fam/w_pcalifornicus_rnaseq_and_orthodb \
> -f TSV,XML,JSON,GFF3 \
> -goterms \
> -i /home/jon/Working_Files/braker2/platanus-allee_illumina_step10.renamed.masked_repeatmodeler-fam/w_pcalifornicus_rnaseq_and_orthodb/braker/augustus.hints.without_asterisk.aa \
> -iprlookup \
> --tempdir home/jon/Working_Files/interproscan/temp_dir \
> --pathways
05/02/2021 00:27:04:234 Welcome to InterProScan-5.48-83.0
05/02/2021 00:27:04:238 Running InterProScan v5 in STANDALONE mode... on Linux
05/02/2021 00:27:49:156 Loading file /home/jon/Working_Files/braker2/platanus-allee_illumina_step10.renamed.masked_repeatmodeler-fam/w_pcalifornicus_rnaseq_and_orthodb/braker/augustus.hints.without_asterisk.aa
05/02/2021 00:27:49:159 Running the following analyses:
[CDD-3.18,Coils-2.2.1,Gene3D-4.2.0,Hamap-2020_05,MobiDBLite-2.0,PANTHER-15.0,Pfam-33.1,PIRSF-3.10,PRINTS-42.0,ProSitePatterns-2019_11,ProSiteProfiles-2019_11,SFLD-4,SMART-7.1,SUPERFAMILY-1.75,TIGRFAM-15.0]
Available matches will be retrieved from the pre-calculated match lookup service.

Matches for any sequences that are not represented in the lookup service will be calculated locally.
05/02/2021 00:28:14:428 Uploaded 52151 unique sequences for analysis
05/02/2021 00:32:19:075 37% completed
05/02/2021 00:32:36:566 62% completed
05/02/2021 00:32:38:949 87% completed
05/02/2021 01:32:39:582 17% completed
05/02/2021 02:32:40:301 21% completed
05/02/2021 03:32:41:887 56% completed
05/02/2021 04:32:42:562 85% completed
2021-02-05 04:54:43,071 [amqEmbeddedWorkerJmsContainer-59] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.RuntimeException: Could not open table 111
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:99) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:80) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:75) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.Level.get(Level.java:131) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.Version.get(Version.java:172) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577) ~[leveldb-0.12.jar:0.12]
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.get(LevelDBStore.java:109) ~[interproscan-persistence-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.getProtein(ProteinDAOImpl.java:150) ~[interproscan-persistence-5.48-83.0.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor135.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy77.getProtein(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:279) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100) ~[interproscan-management-5.48-83.0.jar:?]
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87) ~[interproscan-5.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor103.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.io.FileNotFoundException: /home/jon/Working_Files/interproscan/interproscan-5.48-83.0/home/jon/Working_Files/interproscan/temp_dir/jon-PowerEdge-R910_20210205_002712670_k8ms/kvstore/proteinDB/000111.sst (No such file or directory)
    at java.io.FileInputStream.open0(Native Method) ~[?:?]
    at java.io.FileInputStream.open(FileInputStream.java:219) ~[?:?]
    at java.io.FileInputStream.<init>(FileInputStream.java:157) ~[?:?]
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:126) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:115) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:68) ~[leveldb-0.12.jar:0.12]
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:63) ~[leveldb-0.12.jar:0.12]
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache.get(LocalCache.java:3953) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976) ~[guava-27.1-jre.jar:?]
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4960) ~[guava-27.1-jre.jar:?]
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:92) ~[leveldb-0.12.jar:0.12]
    ... 44 more
2021-02-05 04:54:43,098 [amqEmbeddedWorkerJmsContainer-59] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
java.lang.RuntimeException: Could not open table 111
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:99)
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:80)
    at org.iq80.leveldb.impl.TableCache.newIterator(TableCache.java:75)
    at org.iq80.leveldb.impl.Level.get(Level.java:131)
    at org.iq80.leveldb.impl.Version.get(Version.java:172)
    at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223)
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616)
    at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577)
    at uk.ac.ebi.interpro.scan.persistence.kvstore.LevelDBStore.get(LevelDBStore.java:109)
    at uk.ac.ebi.interpro.scan.persistence.ProteinDAOImpl.getProtein(ProteinDAOImpl.java:150)
    at jdk.internal.reflect.GeneratedMethodAccessor135.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy77.getProtein(Unknown Source)
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.simulateMarshalling(PrepareForOutputStep.java:279)
    at uk.ac.ebi.interpro.scan.management.model.implementations.PrepareForOutputStep.execute(PrepareForOutputStep.java:100)
    at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
    at jdk.internal.reflect.GeneratedMethodAccessor103.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source)
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.FileNotFoundException: /home/jon/Working_Files/interproscan/interproscan-5.48-83.0/home/jon/Working_Files/interproscan/temp_dir/jon-PowerEdge-R910_20210205_002712670_k8ms/kvstore/proteinDB/000111.sst (No such file or directory)
    at java.base/java.io.FileInputStream.open0(Native Method)
    at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:126)
    at org.iq80.leveldb.impl.TableCache$TableAndFile.<init>(TableCache.java:115)
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:68)
    at org.iq80.leveldb.impl.TableCache$1.load(TableCache.java:63)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4960)
    at org.iq80.leveldb.impl.TableCache.getTable(TableCache.java:92)
    ... 44 more
2021-02-05 04:54:43,103 [amqEmbeddedWorkerJmsContainer-59] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :java.lang.RuntimeException: Could not open table 111
2021-02-05 04:54:43,104 [amqEmbeddedWorkerJmsContainer-59] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-02-05 04:54:44,475 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:25] FATAL - Analysis step 13 : Prepare proteins for output of the completed analysis for proteins 27001 to 36000 has failed irretrievably.  Available StackTraces follow.
2021-02-05 04:54:44,478 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:42] FATAL - The JVM will now exit with a non-zero exit status.
2021-02-05 04:54:44,478 [main] [uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster:332] ERROR - Exception thrown by StandaloneBlackBoxMaster: 
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43) ~[interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:157) [interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:475) [interproscan-5.jar:?]
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43)
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:157)
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:475)
InterProScan analysis failed. Exception thrown by StandaloneBlackBoxMaster. Check the log file for details

I tried the interproscan-5.50RC1-83.0-64-bit version you suggested above, but it threw a java heap error. So I switched back to the official interproscan release and set delete.temporary.directory.on.completion=false and created a folder for temp files. There were no files in the temp directory after the above error was thrown.

JonEilers commented 3 years ago

I just realized I forgot a forward slash a the beginning of the temp directory path. Rerunning now and there appears to be temp files written to the temp folder. Will update when it's finished running.

gsn7 commented 3 years ago

@JonEilers your error is slightly different but is again occurring at the point where InterProScan is preparing and writing output results to disk. I have just updated the v5.50RC1-83.0 at https://www.ebi.ac.uk/~nuka/files/interproscan/ and I would advise you test it out and report if you get any errors. You are asking InterProScan to run with 75 threads (-cpu 75), that will definitely result into java heap error unless you change the java heap memory settings (-Xmx) in interproscan.sh to something very high or reduce the number of threads. From your log report, it looks like the lookup is not giving you any performance boost so you are better off disabling it (--disable-precalc )

JonEilers commented 3 years ago

I downloaded the advised version and adjusted the -Xmx to ~300gb of ram and added the --disable-precalc. It's currently running, but I'll be away for the next few days. I'll let you know how it goes when I get back.

srobb1 commented 3 years ago

I have run with using a new temp directory and the delete.temporary.directory.on.completion=false and below is the error I am getting.

I am getting tmp files written. Is there a specific file I should look for?

Thank you!

05/02/2021 12:25:05:820 Welcome to InterProScan-5.50RC1-83.0
05/02/2021 12:25:05:825 Running InterProScan v5 in STANDALONE mode... on Linux
05/02/2021 12:25:12:507 RunID: ginkgo.sgc.loc_20210205_122512377_e2hu
05/02/2021 12:25:37:632 Loading file /scratch/smr/ejr23_trinity_ptychodera_flava/aa/split/Trinity.fasta.transdecoder.clean.pep_part_17.fasta
05/02/2021 12:25:37:636 Running the following analyses:
[CDD-3.18,Coils-2.2.1,Gene3D-4.2.0,MobiDBLite-2.0,PANTHER-15.0,Pfam-33.1,PIRSF-3.10,PRINTS-42.0,ProSitePatterns-2019_11,ProSiteProfiles-2019_11,SFLD-4,SignalP_EUK-4.1,SMART-7.1,SUPERFAMILY-1.75,TIGRFAM-15.0,TMHMM-2.0c]
Pre-calculated match lookup service DISABLED.  Please wait for match calculations to complete...
05/02/2021 12:50:04:432 25% completed
05/02/2021 13:08:55:661 50% completed
05/02/2021 13:20:46:959 75% completed
05/02/2021 13:30:09:920 90% completed
05/02/2021 13:50:08:690 Info: [7001_7024] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
05/02/2021 13:50:08:693 Info: [7001_7024] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
05/02/2021 13:51:56:846 Info: [1_1000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
05/02/2021 13:51:56:847 Info: [1_1000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
2021-02-05 13:51:56,871 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:755) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:533) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:304) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2021-02-05 13:51:56,886 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-02-05 13:51:56,888 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
2021-02-05 13:51:56,888 [amqEmbeddedWorkerJmsContainer-30] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
2021-02-05 13:51:57,216 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:25] FATAL - Analysis step 3 : Prepare proteins for output of the completed analysis for proteins 1 to 1000 has failed irretrievably.  Available StackTraces follow.
2021-02-05 13:51:57,217 [main] [uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError:42] FATAL - The JVM will now exit with a non-zero exit status.
2021-02-05 13:51:57,217 [main] [uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster:341] ERROR - Exception thrown by StandaloneBlackBoxMaster:
java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
    at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43) ~[interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:166) [interproscan-5.jar:?]
    at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:475) [interproscan-5.jar:?]
05/02/2021 13:51:58:025 Info: [1001_2000] pcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
05/02/2021 13:51:58:025 Info: [1001_2000] mcounts  tryCounts:0 maxTrycount:0 maxtotalWaitTime: 0
2021-02-05 13:51:58,040 [amqEmbeddedWorkerJmsContainer-44] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:213] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
org.springframework.orm.jpa.JpaSystemException: org.hibernate.TransactionException: Unable to commit against JDBC Connection; nested exception is javax.persistence.PersistenceException: org.hibernate.TransactionException: Unable to commit against JDBC Connection
    at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:407) ~[spring-orm-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.orm.jpa.DefaultJpaDialect.translateExceptionIfPossible(DefaultJpaDialect.java:127) ~[spring-orm-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:536) ~[spring-orm-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:533) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:304) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy141.executeInTransaction(Unknown Source) ~[?:?]
    at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:197) [interproscan-5.jar:?]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) [spring-jms-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: javax.persistence.PersistenceException: org.hibernate.TransactionException: Unable to commit against JDBC Connection
    at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:60) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:107) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532) ~[spring-orm-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    ... 18 more
Caused by: org.hibernate.TransactionException: Unable to commit against JDBC Connection
    at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:87) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:272) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532) ~[spring-orm-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    ... 18 more
Caused by: org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database is already closed (to disable automatic closing at VM shutdown, add ";DB_CLOSE_ON_EXIT=FALSE" to the db URL) [90121-199]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:617) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:427) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.message.DbException.get(DbException.java:205) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.message.DbException.get(DbException.java:181) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.message.DbException.get(DbException.java:170) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1571) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.jdbc.JdbcConnection.checkClosedForWrite(JdbcConnection.java:1557) ~[h2-1.4.199.jar:1.4.199]
    at org.h2.jdbc.JdbcConnection.commit(JdbcConnection.java:539) ~[h2-1.4.199.jar:1.4.199]
    at jdk.internal.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:390) ~[spring-jdbc-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    at com.sun.proxy.$Proxy29.commit(Unknown Source) ~[?:?]
    at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:81) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:272) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104) ~[hibernate-core-5.4.2.Final.jar:5.4.2.Final]
    at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532) ~[spring-orm-5.1.6.RELEASE.jar:5.1.6.RELEASE]
    ... 18 more
2021-02-05 13:51:58,042 [amqEmbeddedWorkerJmsContainer-44] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:215] ERROR - The exception is :
2021-02-05 13:51:58,043 [amqEmbeddedWorkerJmsContainer-44] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:218] ERROR - 2. The exception is :org.springframework.orm.jpa.JpaSystemException: org.hibernate.TransactionException: Unable to commit against JDBC Connection; nested exception is javax.persistence.PersistenceException: org.hibernate.TransactionException: Unable to commit against JDBC Connection
2021-02-05 13:51:58,043 [amqEmbeddedWorkerJmsContainer-44] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:219] ERROR - StepExecution with errors - stepName: stepPrepareForOutput
13:51:58.058 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
13:51:58.075 [responseMonitorJmsContainer-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table "STEP_EXECUTION" not found; SQL statement:
select stepexecut0_.id as id1_104_0_, stepexecut0_.time_completed as time_com2_104_0_, stepexecut0_.time_created as time_cre3_104_0_, stepexecut0_.exception_first_chunk as exceptio4_104_0_, stepexecut0_.proportion_completed as proporti5_104_0_, stepexecut0_.time_started_running as time_sta6_104_0_, stepexecut0_.state as state7_104_0_, stepexecut0_.step_instance_id as step_ins9_104_0_, stepexecut0_.time_submitted as time_sub8_104_0_, exceptionc1_.step_execution_id as step_exe1_14_1_, exceptionc1_.exception_chunks as exceptio2_14_1_, exceptionc1_.chunk_index as chunk_in3_1_, stepinstan2_.id as id1_105_2_, stepinstan2_.bottom_model as bottom_m2_105_2_, stepinstan2_.bottom_protein as bottom_p3_105_2_, stepinstan2_.step_id as step_id4_105_2_, stepinstan2_.time_created as time_cre5_105_2_, stepinstan2_.top_model as top_mode6_105_2_, stepinstan2_.top_protein as top_prot7_105_2_, dependsupo3_.step_instance_id as step_ins1_107_3_, stepinstan4_.id as depends_2_107_3_, stepinstan4_.id as id1_105_4_, stepinstan4_.bottom_model as bottom_m2_105_4_, stepinstan4_.bottom_protein as bottom_p3_105_4_, stepinstan4_.step_id as step_id4_105_4_, stepinstan4_.time_created as time_cre5_105_4_, stepinstan4_.top_model as top_mode6_105_4_, stepinstan4_.top_protein as top_prot7_105_4_, parameters5_.step_instance_id as step_ins1_106_5_, parameters5_.parameters as paramete2_106_5_, parameters5_.parameters_key as paramete3_5_ from public.step_execution stepexecut0_ left outer join public.exception_chunk exceptionc1_ on stepexecut0_.id=exceptionc1_.step_execution_id inner join public.step_instance stepinstan2_ on stepexecut0_.step_instance_id=stepinstan2_.id left outer join public.step_instance_step_instance dependsupo3_ on stepinstan2_.id=dependsupo3_.step_instance_id left outer join public.step_instance stepinstan4_ on dependsupo3_.depends_upon_id=stepinstan4_.id left outer join public.step_instance_parameters parameters5_ on stepinstan4_.id=parameters5_.step_instance_id where stepexecut0_.id=? [42102-199]
gsn7 commented 3 years ago

@srobb1 this error is slightly different and it doesn't show the file i wanted to inspect but it is also on a different input file from the ones presented previously. I would like to track a bit more the errors that you got before, plus the error above. so unzip this file https://www.ebi.ac.uk/~nuka/files/interproscan/interproscan-lib-patch.zip into the interproscan v5.50RC1-83.0 install directory. then run interproscan against Trinity.fasta.transdecoder.clean.pep_part_1.fasta input file whose errors I have seen previously and also run against Trinity.fasta.transdecoder.clean.pep_part_17.fasta and see if you get any more errors.

I would like to know your system setup, so I can try replicate these problems. let me know the: linux distribution + version java version file system type (the value under column type when you run the command df --print-type /scratch/smr/) file system type for the temp-dir you use

ljudevitluka commented 3 years ago

Hello, I think that I might have the similar error on two different files.

Error code is in the attachement. The some command was used on both files my_interproscan/interproscan-5.48-83.0/interproscan.sh -t p -i Astacus.pep -iprlookup -goterms -pa

Linux distribution and version: CentOS Linux 8 (Core) Java version: openjdk_11.0.1 Files system type: xfs FST for temp: xfs

Should I retry to run the analysis again with Interpro at https://www.ebi.ac.uk/~nuka/files/interproscan/ ? Any help is appreciated :)

All the best,

L.

Error 1.txt Error 2.txt

gsn7 commented 3 years ago

@ljudevitluka your errors are indeed similar and try to run with the version here https://www.ebi.ac.uk/~nuka/files/interproscan/ for your inputs it may be beneficial to disable the match lookup (--disable-precalc) thanks for the details of your system, i will test if i can replicate the problem.

ljudevitluka commented 3 years ago

@gsn7 Thank you! Will do ATM I will also try a previous distribution of Interpro 5.47-82.0 :)

JonEilers commented 3 years ago

@gsn7 The interproscan-5.50RC1-83.0 version seems to have fixed the problem for me. Thanks!

gsn7 commented 3 years ago

@JonEilers glad the new version fixes your problem. since i am unable to replicate this problem on any file system, i included some debug info in the log report, and i would like to see that. also if you can tell me the file system of the temp directory you used (df --print-type /home/jon/Working_Files/interproscan/temp_dir)

ljudevitluka commented 3 years ago

@gsn7 Still in the error area :( Here are the log files.

Best, Luka Error1_new.log Error2_new.log

gsn7 commented 3 years ago

@ljudevitluka the errors you are now getting are Java heap space errors. There are two things you can do to solve these kinds of errors. 1. change the java heap memory settings (-Xmx) in interproscan.sh to something bigger say double the default size and/or 2. split your input sequences file into two

JonEilers commented 3 years ago

@gsn7

OS info

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:    20.04
Codename:   focal

java

java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)

File system info 

df -h --print-type /home/jon/Working_Files/interproscan/temp_dir
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda2      ext4  6.5T  5.2T  1.1T  84% /

I reran it just to make sure, it failed two times and was successful the third time. The new errors are different though? See the logs below. I still have the temp files for all three runs if there is anything in there you need to look at. Error_1 Error_2 Success

ljudevitluka commented 3 years ago

@gsn7 Hello its me again :) Thank you very much for your help! For the first file the analysis with the new version/patch and changing -Xmx worked great. Still waiting for the bigger file, will let you know if something goes wrong. Otherwise thank you again very much!

dence commented 3 years ago

Hi, I'm getting the same error as the original poster when running with InterProScan-5.48-83.0 with 10 cpus on about 12K sequences. Log file is attached here.

When I ran on a much smaller test set with a single cpu, I did not get the error.

The java version loaded with the interproscan module is: openjdk 11.0.2 2019-01-15 OpenJDK Runtime Environment 18.9 (build 11.0.2+9) OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

The OS version is: LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 7.7 (Maipo) Release: 7.7 Codename: Maipo

The file system for the tempdir is: Filesystem Type Size Used Avail Use% Mounted on 10.13.133.11@o2ib,10.13.133.29@o2ib:10.13.133.10@o2ib,10.13.133.28@o2ib:/blue/hpg lustre 3.7P 1.9P 1.8P 51% /blue/kirst

I'm running on a shared compute cluster, so I could ask them to install the updated interproscan referenced in previous posts, but I wanted to check here first.

Regards, Daniel

per_family_iprscan_64335235.txt

gsn7 commented 3 years ago

@dence thanks for reporting this. InterProScan-5.48-83.0 has problems on some file systems when performing intensive input/output tasks. Of course, running on a smaller input set will always work because there isn't much input/out happening compared to a bigger set. Having said that 12K is really not a big set, so we want InterProScan to handle that without problems.

There are improvements in the new release InterProScan-5.50-84.0, as such you should use the new version - https://interproscan-docs.readthedocs.io/en/latest/HowToDownload.html.

dence commented 3 years ago

@gsn7 Thank you for sending me that. I sent that to my sys admin, and I will test it today hopefully.

Thanks, Daniel

gsn7 commented 3 years ago

this bug is now fixed in InterProScan-5.51-85.0