MetaGOflow v1.0 fails to run because of incompatible Prosite binary in IPS 5.57-90.0

[cymon]

MetaGOflow v1.0 aborts with an error on cluster using AMD EPYC 7643 cpus due to the following Prosite binary incompatibility in InterProScan:

28/07/2023 13:53:31:803 Running InterProScan v5 in STANDALONE mode... on Linux
log4j:WARN No appenders could be found for logger (org.apache.activemq.broker.BrokerService).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
28/07/2023 13:53:37:135 RunID: hpc066.a.incd.pt_20230728_135336815_i87a
28/07/2023 13:53:44:194 Loading file /var/lib/cwl/stg0e3d7f9f-ad7f-4ce2-9653-2b028d4e70b2/2000001_3000000.faa
28/07/2023 13:53:44:195 Running the following analyses:
[Pfam-35.0,ProSitePatterns-2022_01,ProSiteProfiles-2022_01,TIGRFAM-15.0]
Pre-calculated match lookup service DISABLED.  Please wait for match calculations to complete...
28/07/2023 15:14:58:144 Uploaded 368019 unique sequences for analysis
2023-07-28 15:17:13,145 [amqEmbeddedWorkerJmsContainer-3] [uk.ac.ebi.interpro.scan.management.model.implementations.RunBinaryStep:199] ERROR - Command line failed with exit code: 1
Command: python3 bin/prosite/runprosite.py data/prosite/2022_01/profile_models /tmp/hpc066.a.incd.pt_20230728_135141098_4rxh//jobPrositeProfiles/000000212001_000000213000.fasta /tmp/hpc066.a.incd.pt_20230728_135141098_4rxh//jobPrositeProfiles/000000212001_000000213000.raw.out bin/prosite/pfsearchV3 -f -o 7 -t 4
Error output from binary:
Error setting affinity!
Error running prosite binary bin/prosite/pfsearchV3

2023-07-28 15:17:13,150 [amqEmbeddedWorkerJmsContainer-3] [uk.ac.ebi.interpro.scan.jms.worker.LocalJobQueueListener:216] ERROR - Execution thrown when attempting to executeInTransaction the StepExecution.  All database activity rolled back.
java.lang.IllegalStateException: Command line failed with exit code: 1
Command: python3 bin/prosite/runprosite.py data/prosite/2022_01/profile_models /tmp/hpc066.a.incd.pt_20230728_135141098_4rxh//jobPrositeProfiles/000000212001_000000213000.fasta /tmp/hpc066.a.incd.pt_20230728_135141098_4rxh//jobPrositeProfiles/000000212001_000000213000.raw.out bin/prosite/pfsearchV3 -f -o 7 -t 4
Error output from binary:
Error setting affinity!
Error running prosite binary bin/prosite/pfsearchV3

This seems to be due to a Prosite binary incompatibility as reported elsewhere with this version of IPS..

The version of IPS in v1.0 is 5.57-90.0 from Aug 2022. Unfortunately, even if this problem has been resolved in newer versions of IPS (latest: June 2023), changes to Docker mean that the current Dockerfile for IPS in MetaGOflow will not compile the newer versions of IPS.

The only way to resolve this for MetaGOflow v.1.0 would be to recompile the Prosite binaries with the newer compatible libraries and rebuild the IPS 5.57-90.0 container. This may not be worth the effort at this stage with more than half of the first EMO BON batch of data already analysed.

However, this should be resolved for MetaGOflow v1.1 (or whatever version we use to analyses the second EMO BON data tranch).

[cymon]

I've made some notes about this issue in a GoogleDoc (request access): InterProScan version needs updating

Here the issue is that the version of IPS in MGF v.1.0.0 uses version 5.57-90.0 which contains an Prosite binary package that is incompatible with AMD EPYC cpus (specifically 7643 cpus (launched 2021), but presumably others with the same architecture). According to this post, the ProSite pftools library needs to be updated to v3.2.11. The latest version of IPS is 5.64-96.0.

This exact error is noted in the IPS documentation:

Additionally on some systems, specially on cluster environments, the following error has been reported Error setting affinity! Error running prosite binary bin/prosite/pfsearchV3 This means the provided binary is not compatible with your system. We have included alternative binaries that you can try, by updating you interproscan.properties file to point to the alternative binaries: binary.prosite.pfscanv3.path=${bin.directory}/prosite/altbin/pfscanV3.noaf binary.prosite.pfsearchv3.path=${bin.directory}/prosite/altbin/pfsearchV3.noaf If the problem persist, you may need to compile those binaries from source, making sure to build without affinity by compiling with the flag cmake -DUSE_AFFINITY=OFF ..

ACTION: We need to make the above changes to IPS version 5.64-96.0 and rebuild the docker image. However, changes to Docker mean that the current Dockerfile will no longer compile - the issue is this command: RUN echo "localhost" > /etc/hostname && echo "127.0.0.1\tlocalhost" > /etc/hosts Using this command is no longer persistent in the resulting image. I think (this is beyond my understanding of Docker) it needs to be set via a parameter in the docker run command, or in the docker-compose config.

[cymon]

Re-opening...