Functional annotation pipeline failing when setting a path to a local copy of interproscan in uppmax

[apfuentes]

Hi @mahesh-panchal,

I am trying to use the nextflow NBIS Functional Annotation pipeline with the uppmax profile, but setting a path to a local copy of the latest version of interproscan (since the current version in Uppmax is two years old). For this, I made a few modifications to the config files, but the pipeline unfortunately fails at the interproscan step.

I downloaded interproscan from here to a local directory in Uppmax . The software was installed following the instructions in the referred website.

Below, I show the modifications made to parts of some of the config files (see the # <<< comments for explanations):

• pipelines-nextflow/FunctionalAnnotation/config/software_packages_uppmax.config:

  process {
  withLabel: blast {
      container = 'quay.io/biocontainers/blast:2.9.0--pl526h3066fca_4'
  }
  withName: interproscan {
      // use Uppmax module
  //module = 'bioinfo-tools:InterProScan/5.30-69.0'  # <<< I commented this line to force looking at the local copy of interproscan
      # <<< Also added the modules below, as they are typically loaded when calling the InterProScan/5.30-69.0 module in Uppmax
      module = 'bioinfo-tools:java/OpenJDK_11.0.2'
      module = 'bioinfo-tools:python3/3.9.5'
      module = 'bioinfo-tools:perl/5.26.2'
      module = 'bioinfo-tools:Phobius/1.01'
      module = 'bioinfo-tools:SignalP/5.0b'
      module = 'bioinfo-tools:tmhmm/2.0c'
  }
  withLabel: 'AGAT' {
      container = 'quay.io/biocontainers/agat:0.5.1--pl526r35_0'
  }
  }

• pipelines-nextflow/FunctionalAnnotation/nextflow.config:

  profiles {
  
  uppmax {
      executor {
          name = 'slurm'
      }
      process {
          scratch = '$SNIC_TMP'
      }
      includeConfig "$baseDir/config/compute_resources.config"
      singularity.enabled = true
      singularity.envWhitelist = 'SNIC_TMP'
      includeConfig "$baseDir/config/software_packages_uppmax.config"
      env.PATH='${PATH}:/proj/<path>/interproscan-5.52-86.0'  # <<< I added this line with the path to the local copy of interproscan
  }
  }

• I made another config file to rewrite the path to the local copy of interproscan and to set the SNIC account, which is called pipelines-nextflow/FunctionalAnnotation/params.config:

  
  # <<< This chunk of code provides again the path to interproscan to the up-max profile, just in case
  profiles {
  
  uppmax {
      executor {
          name = 'slurm'
      }
      process {
          scratch = '$SNIC_TMP'
      }
      includeConfig "$baseDir/config/compute_resources.config"
      singularity.enabled = true
      singularity.envWhitelist = 'SNIC_TMP'
      includeConfig "$baseDir/config/software_packages_uppmax.config"
      env.PATH='${PATH}:/proj/<path>/ensembl_early_release/interproscan-5.52-86.0' 
  }
  }

// Nextflow parameters resume = true process { clusterOptions = '-A snic2021-XX-XX' # <<< I added this line to set the snic compute account // You can also override existing process cpu or time settings here too. }

The pipeline was launched using the conda environment with these commands:

Open screen terminal

screen -S ann

Load Nextflow environment with conda

conda activate nextflow-env

Load modules

module load bioinfo-tools Nextflow/21.04.1

Change NXF_HOME to a place in your project directory (export NXF_HOME=yourprojectfolder)

export NXF_HOME='//work'

Set environment variables

REF_FASTA='/proj//Trachurus_trachurus-GCA_905171665.1-unmasked.fa' GFF_FILE='/proj//Trachurus_trachurus-GCA_905171665.1-2021_03-genes.gff3' PROTEIN_DB='/proj//uniprot_db/uniprot-filtered-organism-Human-2021-07-16.fasta' FUNCT_ANN_PIPELINE='/proj//pipelines-nextflow/FunctionalAnnotation/FunctionalAnnotation.nf' NEXTFLOW_CONFIG='/proj//pipelines-nextflow/FunctionalAnnotation/nextflow_mod.config' OUTPUT_DIR='/proj//functionalAnn_results'

Run the nextflow pipeline

NXF_VER=21.04.1 nextflow run -profile uppmax $FUNCT_ANN_PIPELINE \ --genome $REF_FASTA \ --gff_annotation $GFF_FILE \ --blast_db_fasta $PROTEIN_DB \ --outdir $OUTPUT_DIR

**Questions:**
- Why is the pipeline failing to use the local copy of interproscan?
- Are the modifications made to the config files correct to set the path to the local copy of interproscan?

Thanks

[apfuentes]

Here a snipped of the last lines of the error message printed in screen:

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
        at java.base/java.lang.Thread.run(Thread.java:834)
  java.lang.IllegalStateException: Command line failed with exit code: 1
  Command: python3 bin/prosite/runprosite.py data/prosite/2021_01/profile_models /scratch/21458395/r278.uppmax.uu.se_20210802_122805023_3xnf//jobPrositeProfiles/000000000001_000000001000.fasta /scratch/21458395/r278.uppmax.uu.se_20210802_122805023_3xnf//jobPrositeProfiles/000000000001_000000001000.raw.out bin/prosite/pfsearchV3 -f -o 7 -t 4 
  Error output from binary:
  Error setting affinity!
  Error running prosite binary bin/prosite/pfsearchV3

        at uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep.execute(RunBinaryStep.java:201)
        at uk.ac.ebi.interpro.scan.jms.activemq.StepExecutionTransactionImpl.executeInTransaction(StepExecutionTransactionImpl.java:87)
        at jdk.internal.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy142.executeInTransaction(Unknown Source)
        at uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener.onMessage(LocalJobQueueListener.java:200)
        at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:761)
        at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:699)
        at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
        at java.base/java.lang.Thread.run(Thread.java:834)
  java.lang.IllegalStateException: InterProScan exiting with non-zero status, see logs for further information.
        at uk.ac.ebi.interpro.scan.jms.activemq.NonZeroExitOnUnrecoverableError.failed(NonZeroExitOnUnrecoverableError.java:43)
        at uk.ac.ebi.interpro.scan.jms.master.StandaloneBlackBoxMaster.run(StandaloneBlackBoxMaster.java:167)
        at uk.ac.ebi.interpro.scan.jms.main.Run.main(Run.java:495)
  InterProScan analysis failed. Exception thrown by StandaloneBlackBoxMaster. Check the log file for details

Work dir:
  /crex/proj/snic2020-2-19/private/horse_mackerel/data/00-genome/ensembl_early_release/work/work/29/b51e84025e1c38f0569102ff29bfdd

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

[mahesh-panchal]

If I understand your log snippet correctly, InterProScan is starting, but failing on something. You need to check the InterProScan log for what is failing.

Since the log will be on the scratch partition, it means you need to manually debug the error by running the script on a node.

Here's how: Start an interactive session interactive -A <snic_account> -n <cores> -t <time>. Go to the work directory /crex/proj/snic2020-2-19/private/horse_mackerel/data/00-genome/ensembl_early_release/work/work/29/b51e84025e1c38f0569102ff29bfdd and in there will be some hidden files with the scripts. There are two scripts here of importance. The .command.sh has the bash script. You can edit this to help with debugging, e.g. adding echo, or cat statements. The .command.run script executes the .command.sh in the correct environment. If you look at this script, it will have the slurm directives at the top along with some complicated code for executing the .command.sh. Run the .command.run script using bash .command.run to run the Interproscan process directly on your interactive session (do not submit to slurm again). Also check where you scratch partition is echo $SNIC_TMP. When the process errors out again, find the Interproscan log on the $SNIC_TMP folder and see what the error is.

[mahesh-panchal]

Were you able to find what the error was?

[mahesh-panchal]

Closing this. Feel free to reopen if you still have issues here.