Making sire run on section6

[jmichel80]

I did

sudo apt-get install g++-4.6

Then

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 10 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 20 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.6 10 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 20 sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30 sudo update-alternatives --set cc /usr/bin/gcc sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30 sudo update-alternatives --set c++ /usr/bin/g++

Then can switch to g++4.6 with sudo update-alternatives --config g++

Then module load openmm/6.2 mkdir buildsire cmake ccmake . (Had to manually set paths to opemm in /home/common. Also disabled AVX) make -j 4 make install

Then

mkdir buildwrappers cmake ccmake . (no issues herE) make -j 4 make install

Next issue is broken pip

cd /build_bundled/setuptools-15.0 ~/sire.app/bundled/bin/python3 setup.py install --prefix=/home/$USER/sire.app/bundled/ export PYTHONPATH='/home/$USER/sire.app/bundled/lib/python3.3/site-packages/' cd ../pip-6.0.8 ~/sire.app/bundled/bin/python3 setup.py install --prefix=/home/$USER/sire.app/bundled/

But then ~/sire.app/bundled/bin/pip gives me ImportError: No module named '_ssl'

I have done sudo apt-get install libssl-dev

But still get this error. Maybe I need to repeat buildwrappers from scratch to compile python3.3 now that I have installed libssl-dev ?

[jmichel80]

Ok after installing libssl-dev I had to remove the bundled python3.3 to trigger a recompilation when building wrappers. Then could finish the manual install of setuptools and pip. Then add to sudo apt-get install libncurses5-dev So the lines below would work

~/sire.app/bundled/bin/pip install readline ~/sire.app/bundled/bin/pip install ipython ~/sire.app/bundled/bin/pip install nose

[jmichel80]

Fixed some broken paths in /home/common/Modules/modulefiles/openmm. These were preventing clean build of somd.

[jmichel80]

Completed an epic struggle to make mdtraj work. Solution below: sudo apt-get install libbz2-dev (this must be done before compiling wrappers. If not have to somehow delete python from $HOME/sire.app/bundled/ . Easiest thing to to is to remove the binary install and buildwrapper and start again.

Note that this means redoing all the above pip installs.

Then when you are back at this stage ~/sire.app/bundled/bin/pip install numpy ~/sire.app/bundled/bin/pip install cython

The procedure below is more complex than a sudo apt-get install blas/lapack but enables to build in support for blas lapack without root permissions

mkdir -p ~/src/ cd ~/src/ wget http://www.netlib.org/blas/blas.tgz tar xzf blas.tgz cd BLAS gfortran -O3 -std=legacy -m64 -fno-second-underscore -fPIC -c .f ar r libfblas.a .o ranlib libfblas.a rm -rf *.o cp libfblas.a ~/sire.app/bundled/lib/libblas.a export BLAS=~/sire.app/bundled/lib/libblas.a

mkdir -p ~/src cd ~/src/ wget http://www.netlib.org/lapack/lapack.tgz tar xzf lapack.tgz cd lapack-*/ cp INSTALL/make.inc.gfortran make.inc # on Linux with lapack-3.2.1 or newer

HERE you must edit make.inc and replace -frecursive with -fPIC for the lines starting with OPS and NOOPTS

make lapacklib cp liblapack.a ~/sire.app/bundled/lib/ export LAPACK=~/sire.app/bundled/lib/

~/sire.app/bundled/bin/pip install scipy ~/sire.app/bundled/bin/pip install mdtraj

julien@node009:~/src/lapack-3.5.0$ ~/sire.app/bin/python "/users/julien/sire.app/share/Sire/scripts/python.py" Starting /home/julien/sire.app/bin/python: number of threads equals 16 WARNING: IPython History requires SQLite, your history will not be saved Python 3.3.3 (default, Jul 7 2015, 12:23:41) Type "copyright", "credits" or "license" for more information.

IPython 3.2.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

In [1]: import mdtraj

In [2]:

BUT

julien@node005:~$ ~/sire.app/bin/python "/home/julien/sire.app/share/Sire/scripts/python.py" Starting /home/julien/sire.app/bin/python: number of threads equals 32 WARNING: IPython History requires SQLite, your history will not be saved Python 3.3.3 (default, Jul 7 2015, 12:23:41) Type "copyright", "credits" or "license" for more information.

IPython 3.2.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. i

In [1]: import mdtraj

ImportError Traceback (most recent call last)

in () ----> 1 import mdtraj /home/julien/sire.app/bundled/lib/python3.3/site-packages/mdtraj/**init**.py in () 49 from mdtraj.core import element 50 from mdtraj._rmsd import rmsd ---> 51 from mdtraj._lprmsd import lprmsd 52 from mdtraj.core.topology import Topology 53 from mdtraj.geometry import * _lprmsd.pyx in init _lprmsd (mdtraj/rmsd/_lprmsd.cpp:7966)() /home/julien/sire.app/bundled/lib/python3.3/site-packages/scipy/spatial/**init**.py in () 89 from .kdtree import * 90 from .ckdtree import * ---> 91 from .qhull import * 92 from ._plotutils import * 93 ImportError: libgfortran.so.3: cannot open shared object file: No such file or directory In [2]: So cannot use mdtraj on the compute nodes because of the missing library. The following does the trick cp /usr/lib/x86_64-linux-gnu/libgfortran.so.3 ~/sire.app/bundled/lib/ julien@node005:~$ ~/sire.app/bin/python "/home/julien/sire.app/share/Sire/scripts/python.py" Starting /home/julien/sire.app/bin/python: number of threads equals 32 WARNING: IPython History requires SQLite, your history will not be saved Python 3.3.3 (default, Jul 7 2015, 12:23:41) Type "copyright", "credits" or "license" for more information. IPython 3.2.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: import mdtraj In [2]: But it's ridiculous I have to do all of this to make mdtraj work. Another probable fix is to sudo apt-get gfortran on every node but that won't work if we deploy the code on a cluster where we do not have sudo rights. On a good note, the nautilus tests now pass julien@node005:~/sire.app/test/Tools$ ~/sire.app/bundled/bin/nosetests test_nautilus.py (...) Ran 7 tests in 199.378s OK

[ppxasjsm]

Wow this does look pretty terrible! This really is not a good way to make deployment easy. We should really look into using anaconda. Maybe things would have been easier installing it that way. According to the website it should be as easy as doing this: $ conda config --add channels http://conda.binstar.org/omnia

$ conda install mdtraj

[jmichel80]

We need python3 support. It looks like omnia doesn't support this yet

http://www.omnia.md/install/

[ppxasjsm]

Ah true! I hadn't thought of that.

[ppxasjsm]

Does ccmake work for you, to configure Sire? I don't seem to get expected behaviour of it.

[jmichel80]

Can you elaborate on what you mean by 'expected behaviour' ? I usually follow cmake by ccmake to fix for instance errors in detecting the openmm library folder, or to tweak other settings (AVX etc...).

On Thursday, July 16, 2015, ppxasjsm notifications@github.com wrote:

Does ccmake work for you, to configure Sire? I don't seem to get expected behaviour of it.

— Reply to this email directly or view it on GitHub https://github.com/michellab/Cluster/issues/3#issuecomment-121982539.

---

Dr. Julien Michel, Royal Society University Research Fellow Room 263, School of Chemistry Joseph Black Building, University of Edinburgh David Brewster Road

Edinburgh

EH9 3FJ United Kingdom phone: +44 (0)131 650 4797

http://www.julienmichel.net/

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

[ppxasjsm]

Just a quick question regarding mdtraj/nautilus. You were concerned about the fortran libraries being present on a cluster for deployment. I think I would expect blas/lapack and fortran libraries to be present on a well maintained cluster. Also you would have to install fortran libraries manually, if you don't have sudo rights on a cluster... Let's hope there will be a python3 support for the omnia install route soon. That would make things a lot easier I would suspect.

[jmichel80]

Or maybe a conda python install would do, but that needs some testing. I will research this as part of the work I'm doing to prepare a nautilus release.

On Friday, July 17, 2015, ppxasjsm notifications@github.com wrote:

Just a quick question regarding mdtraj/nautilus. You were concerned about the fortran libraries being present on a cluster for deployment. I think I would expect blas/lapack and fortran libraries to be present on a well maintained cluster. Also you would have to install fortran libraries manually, if you don't have sudo rights on a cluster... Let's hope there will be a python3 support for the omnia install route soon. That would make things a lot easier I would suspect.

— Reply to this email directly or view it on GitHub https://github.com/michellab/Cluster/issues/3#issuecomment-122239929.

---

Dr. Julien Michel, Royal Society University Research Fellow Room 263, School of Chemistry Joseph Black Building, University of Edinburgh David Brewster Road

Edinburgh

EH9 3FJ United Kingdom phone: +44 (0)131 650 4797

http://www.julienmichel.net/

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

[ppxasjsm]

Readline issues: I have successfully installed the dev branch of Sire, but I can't install readline. You managed to do this without any problems before. I get this error from pip install which is not very informative to me:

    /usr/bin/gcc-4.6 -pthread -shared -Wl,-rpath=$ORIGIN/../lib build/temp.linux-x86_64-3.3/Modules/3.x/readline.o readline/libreadline.a readline/libhistory.a -L. -lncurses -lpython3.3m -o build/lib.linux-x86_64-3.3/readline.cpython-33m.so

    /usr/bin/ld: cannot find -lpython3.3m

    collect2: ld returned 1 exit status

    Starting /home/ppxasjsm/sire.app/bin/python: number of threads equals 16

    error: command '/usr/bin/gcc-4.6' failed with exit status 1

    ----------------------------------------
    Command "/home/ppxasjsm/sire.app/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-sjwxs3/readline/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-abrwn5-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-sjwxs3/readline

On the workstation this is no problem, and googling didn't get me anywhere either!

[jmichel80]

I am trying to recompile sire on the cluster. I saw that the default gcc was 4.6 but g++ was 4.8. I switched both to 4.6 with update-alternatives before compiling. Could mixing versions creates such errors downstream ?

[ppxasjsm]

Nope, this was my attempt to see if mixed versions would actually allow me to recompile things and let me allow to install readline, since on my workstation I (unintentionally) was using mixed versions. The current work around is to copy and the compiled sire version from my workstation to the cluster. Then numpy runs fine.

[ppxasjsm]

Did you have any problems installing readline?

[jmichel80]

I didn't re-install sire, just updated the git sources and recompiled. Would you like me to delete my sire.app binary folder and do a fresh install of all python modules to see if I also encounter issues with readline ? I could try next week if necessary.

---

Dr. Julien Michel, Royal Society University Research Fellow Room 263, School of Chemistry Joseph Black Building, University of Edinburgh David Brewster Road

Edinburgh

EH9 3FJ United Kingdom phone: +44 (0)131 650 4797

http://www.julienmichel.net/

On Fri, Aug 7, 2015 at 3:10 PM, ppxasjsm notifications@github.com wrote:

Did you have any problems installing readline?

— Reply to this email directly or view it on GitHub https://github.com/michellab/Cluster/issues/3#issuecomment-128710509.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

[ppxasjsm]

This might need to be tested. Probably not worth wasting time on it right now though.

[ppxasjsm]

I think this is nolonger relevant and I'll close the issue.