search

softwaresaved / visualisation

Web-based visualisation of Institute data.Contact: @mikej888
http://softwaresaved.github.io/visualisation
GNU General Public License v2.0
2 stars 1 forks source link
metrics policy service visualisation

readme

Software Sustainability Institute web-based data visualisation

Visualisations of The Software Sustainability Institute data.

To view, visit http://softwaresaved.github.io/visualisation.

Requesting a visualisation for a data set (Institute staff)

To request that a data set be added to the visualisations:

Note: Any vague suggestions for which data does not yet exist, or does exist but not as a Google Sheet, can be added as a comment to the general issue, What data should we visualise?.

Deploy visualisations

Get source code

Fork and clone this repository

$ git clone http://USERNAME@github.com/USERNAME/visualisation.git
$ cd visualisation
$ git remote add ssi  http://USERNAME@github.com/softwaresaved/visualisation.git

Get Python 3

The Anaconda version of Python is recommended.

Install Python packages

$ pip install -r requirements.txt

This installs (if not already installed):

View visualisations within GitHub pages

GitHub pages renders web site content in gh-pages branches of repositories hosted on GitHub.

When you push changes to the gh-pages branch of your cloned repository on GitHub, you can view the web site at http://USERNAME.github.io/visualisation/

View visualisations within Python web server

Some browsers (e.g. Google Chrome) won't render JavaScript in HTML that is loaded into the browser via, for example, a file:// URL.

Python 3 has a simple web server you can use to serve the pages. To start the web server:

$ cd visualisation
$ python -m http.server

Visit http://localhost:8000/ to see index.html.

Deploy under Apache 2 web server

Apache is a popular web server.

These instructions assume you have sudo access to install and configure software (or a local system administrator can do this for you):

$ sudo su -

Install Apache 2 on Ubuntu 14.04.3 LTS Trusty Tahr

Install:

$ apt-get install -y apache2 apache2-utils
$ apache2 -v
Server version: Apache/2.4.7 (Ubuntu)
Server built:   Jan 14 2016 17:45:23
$ service apache2 status
 * apache2 is running

Install Apache 2 on Scientific Linux 6.6 (Carbon)

Install:

$ apt-get install -y httpd
$ /usr/sbin/httpd -v
Server version: Apache/2.2.15 (Unix)
Server built:   Aug 24 2015 11:20:01
$ chkconfig httpd on
$ /etc/init.d/httpd status
httpd (pid  18161) is running...

Install Apache 2 on Scientific Linux 7.1 (Nitrogen)

Install:

$ apt-get install -y httpd
$ /usr/sbin/httpd -v
Server version: Apache/2.4.6 (Scientific Linux)
Server built:   Jul 23 2014 05:03:32
$ systemctl restart httpd.service
$ systemctl status httpd.service
$ systemctl enable httpd.service
$ systemctl status httpd.service
httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
   Active: active (running) since Mon 2016-03-07 13:02:03 GMT; 19s ago
n Main PID: 14675 (httpd)

Clone the repository

Clone:

$ cd /var/www/html
$ git clone https://github.com/softwaresaved/visualisation

Set permissions

$ chown -R www-data:www-data visualisation
$ chown -R apache:apache visualisation

View visualisations

Visit http://127.0.0.1/visualisation/ to see index.html.

Update data (Institute staff)

Updates to data in data/ can be downloaded from the Institute's Google Sheets and private GitHub repositories.

Set up Google API authentication

You need to set up Google API authentication to access the Institute's Google Sheets as they are private.

Create Google API project:

Enable Google Sheets API:

Create OAuth2 credentials:

Run a data download script to add the Google API authentication files to your local machine:

Download data:

$ python src/download_sheets.py --noauth_local_webserver --sheets sheets.yml

When you run this script for the first time, with the given flag, Google's authentication component will ask you to give your permission for the use of your credentials to access Google Sheets to which you have access. It does this by asking you to go to a Google Accounts link to get a verification code and enter that code. For example:

Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?client_id=12345-678910.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&access_type=offline&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fspreadsheets.readonly

Enter verification code:

Using a web browser:

Enter the verification code to successfully authenticate. For example:

Enter verification code: ABCDEF1234567890

You should then see a message like:

Authentication successful.
Storing credentials to /home/someuser/credentials/sheets.googleapis.com-python-quickstart.json

When you run any of the download scripts, you will not be asked to authenticate again.

Download updated Google Sheets data

Run:

$ python src/download_sheets.py --noauth_local_webserver --sheets sheets.yml

Create GitHub personal access token

You need to set up a GitHub personal access token as some of the Institute's GitHub repositories are private.

Within GitHub:

Download updated GitHub data

Run:

$ python src/download_repository_files.py --token <YOUR_GITUHUB_TOKEN> --repositories repositories.yml

As an alternative to the command-line flag, a GITHUB_TOKEN environment variable can be set e.g.

$ GITHUB_TOKEN=<YOUR_GITHUB_TOKEN> python src/download_repository_files.py --repositories repositories.yml

Development

Introduction to HTML, CSS, JavaScript and D3:

Browser developer tools:

Comments:

Logging:

Google Chrome developer tools:

Google Sheets and OAuth2 APIs:

Copyright and licence

Third-party code

Google Sheet and OAuth2 Python example

Automattic Color.js

Bootstrap 3.3.6

D3 4.2.1

D3 horizontal bar chart

D3 pie chart

D3 circle packing

jQuery 2.2.1

Underscore 1.8.3