psipred buffer overflow detected

cdienem commented 5 months ago

Dear RoseTTAFold team,

when running the protein example after fixing issues #32 and #37 , I run into an error with psipred:

Running PSIPRED buffer overflow detected : ======= Backtrace: ========= /lib64/libc.so.6(fortify_fail+0x37)[0x7f282a2d /lib64/libc.so.6(+0x1166f2)[0x7f282a2d26f2] /lib64/libc.so.6(__fgets_chk+0x129)[0x7f282a2d2a19] psipred[0x401390] psipred[0x40158a] /lib64/libc.so.6(libc_start_main+0xf5)[0x7f28 psipred[0x400a89] ======= Memory map: ======== 00400000-00402000 r-xp 00000000 fd:02 23127977 00601000-00602000 r--p 00001000 fd:02 23127977 00602000-00603000 rw-p 00002000 fd:02 23127977 00603000-006d3000 rw-p 00000000 00:00 0 02265000-02286000 rw-p 00000000 00:00 0 7f282a1bc000-7f282a37f000 r-xp 00000000 fd:00 33588779 7f282a37f000-7f282a57f000 ---p 001c3000 fd:00 33588779 7f282a57f000-7f282a583000 r--p 001c3000 fd:00 33588779 7f282a583000-7f282a585000 rw-p 001c7000 fd:00 33588779 7f282a585000-7f282a58a000 rw-p 00000000 00:00 0 7f282a58a000-7f282a68b000 r-xp 00000000 fd:00 33588787 7f282a68b000-7f282a88a000 ---p 00101000 fd:00 33588787 7f282a88a000-7f282a88b000 r--p 00100000 fd:00 33588787 7f282a88b000-7f282a88c000 rw-p 00101000 fd:00 33588787 7f282a88c000-7f282a8ae000 r-xp 00000000 fd:00 33588491 7f282aa8a000-7f282aa8d000 rw-p 00000000 00:00 0 7f282aa8f000-7f282aa93000 r--p 00000000 fd:02 541974413 7f282aa93000-7f282aaa5000 r-xp 00004000 fd:02 541974413 7f282aaa5000-7f282aaa8000 r--p 00016000 fd:02 541974413 7f282aaa8000-7f282aaa9000 r--p 00019000 fd:02 541974413 7f282aaa9000-7f282aaaa000 rw-p 0001a000 fd:02 541974413 7f282aaaa000-7f282aaad000 rw-p 00000000 00:00 0 7f282aaad000-7f282aaae000 r--p 00021000 fd:00 33588491 7f282aaae000-7f282aaaf000 rw-p 00022000 fd:00 33588491 7f282aaaf000-7f282aab0000 rw-p 00000000 00:00 0 7fff68f21000-7fff68f42000 rw-p 00000000 00:00 0 7fff68ffe000-7fff69000000 r-xp 00000000 00:00 0 ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 psipred terminated 4577] 2a1de555] /home/X/opt_test/conda_envs/RFAA/bin/psipred /home/X/opt_test/conda_envs/RFAA/bin/psipred /home/X/opt_test/conda_envs/RFAA/bin/psipred [heap] /usr/lib64/libc-2.17.so /usr/lib64/libc-2.17.so /usr/lib64/libc-2.17.so /usr/lib64/libc-2.17.so /usr/lib64/libm-2.17.so /usr/lib64/libm-2.17.so /usr/lib64/libm-2.17.so /usr/lib64/libm-2.17.so /usr/lib64/ld-2.17.so /home/X/opt_test/conda_envs/RFAA/lib/libgcc_s.so.1 /home/X/opt_test/conda_envs/RFAA/lib/libgcc_s.so.1 /home/X/opt_test/conda_envs/RFAA/lib/libgcc_s.so.1 /home/adminc/opt_test/conda_envs/RFAA/lib/libgcc_s.so.1 /home/X/opt_test/conda_envs/RFAA/lib/libgcc_s.so.1 /usr/lib64/ld-2.17.so /usr/lib64/ld-2.17.so [stack] [vdso] [vsyscall]

The system is CentOS7. Does psipred come pre-compiled? If its compiled during installation of RoseTTAFOld-all-atom, the local gcc verison is 4.8.5.

Please let me know if you need more information.

felixmb1 commented 5 months ago

I get the same issue and have tried fixes from previous rosettafold release https://github.com/Sabryr/ProteinFolding/issues/3 but it still doesn't work

cdienem commented 5 months ago

Alright, I went through many issues so I will summarize what has worked for me to get rid of all of them.

The main issues are:

input/make_ss.sh is not executable
UniRef is referenced witht he wrong version compared to the one recommended in the installation description
RFAA looks for UniRef at a location called "uniclust" and not "UniRef..."
BLAST that is shipped does not work, needs to be updated
PSIPRED that is shipped may not work on particular systems, should be compiled at install time

After fixing these, I successfully ran the protein example.

How to fix the above? First, install RFAA as described. Then apply the following fixes:

# activate your conda environment
conda activate RFAA

# Define where RFAA is installed
RFAA=path/to/rosettafold-all-atom
cd $RFAA

# The script input_prep/make_ss.sh needs to be executable
chmod +x input_prep/make_ss.sh

# The UniRef version in make_msa.sh is wrong. Rename to the correct UniRef version
sed -i "s|UniRef30_2021_06|UniRef30_2020_06|g" make_msa.sh

# The uniref database is expected to be "uniclust" and not "UniRef..." so symlink it
ln -s UniRef30_2020_06 uniclust

# Get new blast version as the one shipped with the installation does not work and make it available in input_prep/makse_ss.sh
wget https://ftp.ncbi.nlm.nih.gov/blast/executables/legacy.NOTSUPPORTED/2.2.26/blast-2.2.26-x64-linux.tar.gz
mkdir -p blast-2.2.26
tar -xf blast-2.2.26-x64-linux.tar.gz -C blast-2.2.26
cp -r blast-2.2.26/blast-2.2.26/ blast-2.2.26_bk
rm -r blast-2.2.26
mv blast-2.2.26_bk/ blast-2.2.26

# Insert sourcing of the correct blast (pay attention to the location delivered by $RFAA)
new_line="export BLASTMAT=$RFAA/blast-2.2.26/data/"
sed -i "3 a$new_line" $RFAA/input_prep/make_ss.sh

# Recompile psipred in case the shipped one runs into a buffer overflow (I guess it would be better to always compile during installation)
cd $RFAA
wget http://bioinfadmin.cs.ucl.ac.uk/downloads/psipred/psipred.4.02.tar.gz
tar xf psipred.4.02.tar.gz
cd psipred/src
make
make install

chmod +x psipred
chmod +x psipass2

# Replace the psipred binaries within the conda environment with the recompiled ones (symlink them)

mv $CONDA_PREFIX/bin/psipred $CONDA_PREFIX/bin/psipred.orig
mv $CONDA_PREFIX/bin/psipass2 $CONDA_PREFIX/bin/psipass2.orig

ln -s $RFAA/psipred/bin/psipred $CONDA_PREFIX/bin/psipred
ln -s $RFAA/psipred/bin/psipass2 $CONDA_PREFIX/bin/psipass2

So this should be easy to implement now for the devs.

NiklasTR commented 5 months ago

Took me steps 1, 2, and 3 to find this issue - great work. Worth a PR

baker-laboratory / RoseTTAFold-All-Atom

psipred buffer overflow detected #72