Deploy cctbx including new MolProbity on AWS Graviton2 processors

[russell-taylor]

sudo apt update; sudo apt upgrade
sudo apt install python build-essential libgl1 freeglut3-dev xvfb
mkdir -p ~/src/; cd ~/src; git clone https://github.com/cctbx/cctbx_project
mkdir -p ~/build/cctbx; cd ~/build/cctbx
python ~/src/cctbx_project/libtbx/auto_build/bootstrap.py --use-conda --python=36

(For multi-CPU instances with sufficient memory, can use --nproc=N where N is the number of processors)

[russell-taylor]

For AWS Graviton processors, we cannot use conda as is because there is not an associated package to use

Trying to build without conda: python ~/src/cctbx_project/libtbx/auto_build/bootstrap.py --python=36

Results in the following errors after OpenSSL is successfully installed:

Installing Python3...
  log file is /home/ubuntu/build/cctbx/base_tmp/Python3_install_log
  getting package Python-3.7.2.tgz...
    downloading from https://gitcdn.link/repo/dials/dependencies/master/Python-3.7.2.tgz : 22361 kB
  installing Python-3.7.2.tgz...
Traceback (most recent call last):
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 1682, in <module>
    installer(args=sys.argv, log=sys.stdout)
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 256, in __init__
    self.build_dependencies(packages=packages)
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 782, in build_dependencies
    getattr(self, 'build_%s'%i)()
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 800, in build_python
    return self.build_python3()
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 938, in build_python3
    log=log, cwd=python_dir, shell=False, env=environment)
  File "modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 359, in call
    return call(args, log=log, verbose=self.verbose, **kwargs)
  File "/home/ubuntu/build/cctbx/modules/cctbx_project/libtbx/auto_build/installer_utils.py", line 55, in call
    raise RuntimeError("Call to '%s' failed with exit code %d" % (args, rc))
RuntimeError: Call to '['make', 'install', '-j', '1']' failed with exit code 2
Process failed with return code 1

[russell-taylor]

There was a Python 3.7 set of directories in the base/bin directory. trying rerunning with --python=37

This fails with the same error.

[russell-taylor]

(Consider using the Miniconda3-latest-Linux-armv7l.sh instead of the i86 version that it tries to install; determine how to tell) Nope: This build is for Raspberry Pi 2 running Raspbian Wheezy and the Graviton is not ARMv7l.

[russell-taylor]

The aws-graviton-getting-started page reports that the following packages are needed to install Python from source:

sudo apt-get install python3-pip python3-dev libblas-dev gfortran liblapack-dev
python3 -m pip install --user --upgrade pip

After installing these, retried python ~/src/cctbx_project/libtbx/auto_build/bootstrap.py --python=36

Note: I expect that some of those packages may not be needed for our case: libblas-dev gfortran liblapack-dev

[russell-taylor]

Adding those packages and pip installs did not fix the problem -- we get the same error message.

[russell-taylor]

Leaving this to CCTBX folks.