Build new instance w/o docker

AlexanderPico commented 8 years ago

Following instructions here: http://wiki.cytoscape.org/AppStore on new VM at UCSD.

Notes

Create jeff account
Installed LAMP

apt-get install apache2
apt-get install libapache2-mod-wsgi
apt-get install mysql-server
apt-get install php5 libapache2-mod

Installed Python 2.6 (ended up with 2.7)

add-apt-repository ppa:fkrull/deadsnakes
apt-get update
apt-get install python2.6 python2.6-dev

_Note: Looks like I've got both 2.6 and 2.7 installed... Could be a problem later. Looks like 2.7 is the active one..._

Installed xapian

apt-get install python-xapian
apt-get install libxapian-dev

Installed xapian-binding from source

wget http://oligarchy.co.uk/xapian/1.2.13/xapian-bindings-1.2.13.tar.gz
tar -xzvf xapian-bindings-1.2.13.tar.gz
cd xapian-bindings-1.2.13
./configure --with-python

_Error: Python.h not found!_

Installed regular python-dev to address error (though might be dependent on Python 2.7 now?

apt-get install python-dev

Second attempt to build xapian-bindings

./configure --with-python

_configure: WARNING: Xapian library is version 1.2.16 but the bindings are version 1.2.13 - we strongly recommend using matching versions._

Installed image libs

apt-get install libjpeg8
apt-get install libjpeg-dev
apt-get install libjpeg8-dev
apt-get install libpng-dev

Installed GeoIP

apt-get install php5-geoip
apt-get install libgeoip-dev
pip install geopip
apt-get install libgeoip1

_Tried all of these in vain attempt to generate /usr/lib/libGeoIP.so.1 installed (needed later). In the end, I found it at /usr/lib/x86_64-linux-gnu/libGeoIP.so.1 and I'm not sure which command above generated it._

Installed python packages listed in wiki docs using pip install

easy_install pip
pip install django
apt-get install libmysqlclient-dev
pip install mysql-python
pip install pillow
pip install django-social-auth
pip install IPython

Code

Clone code git clone http://github.com/cytoscape/appstore .
Followed instructions for conf dir files. Ran django-admin startproject cyappstore to generate a temporary settings.py with a new SECRET_KEY to add to apikeys.py. Copied all other values from previous instance of CyAppStore. _Make sure dbs.py info matches the actual mysql info_
Installed mvn in /home/jeff for mnv.py

mkdir cyappstore-pkgs
cd cyappstore-pkgs
wget http://www.carfab.com/apachesoftware/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
tar xzvf apache-maven-3.3.9-bin.tar.gz

Copied apache config for appstore.conf and updated python dir path to use 2.7. Also made logs dir mkdir /var/www/CyAppStore/logs. _NOTE: the 2.6 vs. 2.7 issue is also expressed here_
Then enabled conf and reloaded

sudo a2ensite appstore.conf
service apache2 reload

Also instanced and updated maven-app-repo-settings.xml with same info as in previous site
Followed instructions for geolite dir

_Waiting on creating cronjobs for later..._

Loaded data from an old mysqldump

create user 'cyappstore'@'localhost' identified by 'appsrule';
grant all on cyappstore.* to 'cyappstore'@'localhost';
create database cyappstore;
use cyappstore;
source cyappstore-160414.sql;

_Will update with latest backup just prior to going live_

Test

Dependencies python external_scripts/test_dependencies.py ***FAILS: python version (ok); django version (ok); JAVA_HOME (fix and try again)

apt-get install default-jre
update-alternatives --config java
vi /etc/environment
JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java"

source /etc/environment

GeoIP python manage.py test_geoip _We have a problem! importlib call changed in newer version of django which is paired with newer version of python. Press ahead or get old stable versions? Let's try old..._

apt-get install python-virtualenv
virtualenv --python=/usr/bin/python2.6 /var/www/CyAppStore
source bin/activate
pip install Django==1.4.5

And reinstall all the pip's above now in the python2.6 env and repeat the xapian wget and build in this virtualenv as well...

Now both dependency test run OK!

Temporary domain setting

vi /etc/hosts
132.239.8.177   apps-dev.cytoscape.org
vi /etc/apache2/sites-enabled/appstore.conf
ServerName  localhost
a2dissite 000-default

_UNDO these after domain is properly set_

Addressing missing sys.path for settings.py

add2virtualenv /var/www/CyAppStore

AlexanderPico commented 8 years ago

User and admin logins are broken.
Cronjobs need to be added
Much testing needed...

AlexanderPico commented 8 years ago

Undid temporary domain settings in /etc/hosts and enabled-sites, now that apps-test.cytoscape.org is mapped to the IP.

Also went to google oauth site to verify this new domain: https://console.developers.google.com/apis/credentials/domainverification?project=api-project-453052878726&authuser=2 Required adding a TXT record via the DNS provider (see Benno). Also edited existing credentials to include this new site: Credentials>Web client 2>Authorized redirect URIs. Note: This step is only required because we are using a new domain name. It wouldn't be needed if we were reusing an existing subdomain, like "apps" or "apps-dev" (or "apps-test" in the future). Also note: adding a new redirect to the existing credentials alone might have been enough; not sure if domain verification was necessary.

AlexanderPico commented 8 years ago

Added cronjobs via jeff's crontab -e. Commented out backup job until this instance is designated as the live instance. IOW, only one instance should write backups and ship to pointer for storage.

AlexanderPico commented 8 years ago

Updated permissions: Set owner and group to www-data for all .pyc file and media dir.

Everything is working except:

admin page
pending apps

Both issues may have to do with python2.7 being invoked... or not.

AlexanderPico commented 8 years ago

Fix to admin page (via tip from Samad): Added path to admin template from myvirtualenv Python2.6 to settings.py around line 101:

TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
    filejoin(SITE_DIR, 'templates'),
    filejoin(SITE_DIR, 'lib/python2.6/site-packages/django/contrib/admin/templates'),
)

AlexanderPico commented 8 years ago

To direct system to use python2.6 when running Django, I added the following line to /etc/apache2/mods-enabled/wsgi.conf

        WSGIPythonPath /var/www/CyAppStore/lib/python2.6/site-packages

And then restarted apache.

AlexanderPico commented 8 years ago

Fixed pending apps page by unpacking fresh copy of media dir, setting group to www-data and deleting everything already in the media/pending_releases dir. Apparently, some old file there was messing it up.

DONE!

AlexanderPico commented 8 years ago

backup.sh script is working. But note that the scp of the media files to pointer.ucsf.edu is very, very slow. Might pick a UCSD backup server instead...

AlexanderPico commented 8 years ago

Reset file permissions to match information in repo files by performing a git checkout on all files showing up in git status with only permission mode changes.

I didn't know this information was contained in the repo. Cool!

cytoscape / appstore