Introduction

This is a repository to accompany the GWAS Diversity Monitor, currently maintained as part of the Leverhulme Centre for Demographic Science. The dashboard can be found at:

The frontend is written in D3 4.0 (a JavaScript library for visualizing data with HTML, SVG, and CSS) with the backend hosted in Flask (a lightweight WSGI web application framework written in Python). Grateful attributions regarding data are made to the EMBL-EBI GWAS Catalog. In summary, the dashboard visualizes a systematic interactive review of all GWAS published to date. This repo can be cloned and ran on-the-fly to generate a server on localhost as required. The dashboard and associated summary statistics check daily for updates from the Catalog and update with new releases. The cron adapts a couple of functions from our Scientometric Review of all GWAS

Prerequisites

As a pre-requisite to running this locally, you will need a working Python 3 installation with all of the necessary dependencies detailed in requirements.txt (generated by pipreqs). We strongly recommend virtual environments and Anaconda.

Running the Code

This server is operating system independent (through the os module) and should work on Windows, Linux and macOS all the same. To run: clone this directory, cd into the directory, download the data with "python app/generate_data.py", and serve the project by simply calling the gwasdiversitymonitor.py file. "python gwasdiversitymonitor.py". This will serve the project to port 5000 on your localhost (via Flask).

To do this run "python -m venv virtualenv" from the root of the project. This will create a directory called "virtualenv". Navigate into virtualenv/bin and run "pip install -r requirements.txt" to install the requirements of the project inside your new virtual environment. Then run the project from the root of the project (above the virtualsnv/) with ./virtualenv/bin/python gwasdiversitymonitor.py.

Structure

The root of the folder contains routes.py, which is the main file which runs the application. This file registers paths inside the application, creates appropriate variables, and passes them into templates (html files) which renders the page. This is our Controller in an MVC framework.

Each call to @app.route('path') defines a path and almost all of these just return templates for pages. The exception is @app.route("/getCSV/") which is handling the data download for all cases and returns a file response. The other exception in this file is, at the top, @app.context_processor. This injects some functionality into our flask templates for us. It is used to give us a user_agent checker (check what device/browser is accessing the application) check the state of the cookie policy, and inject the google analytics key from the config file.

The other important file here is DataLoader.py. This is a simple file containing a series of helper functions that route.py uses to load and reshape the data from the wrangled csv's into a shape that is workable with d3.

Data: pulls, wrangles and creates all data used in this project.

Static: This subfolder contains all of the assets for the application. CSS/Sass, fonts, images and js will all be found here. The one we care most about is the js. There is a script.js file which contains some global functions and then a file for each graph.

Each graph js file contains a function, which is then called from the global template, to instantiate the graph. The global filters recall these functions to redraw the graphs with the new filter state. "internal" filters, eg. year and parent term for each tile, are handled by event handlers inside each graph file.

Docker Deployment

To launch the app using the Docker deployment, you must first install Docker Compose.

The Docker deployment for this application uses three containers that are defined in docker-compose.yml:

1. gwas_nginx: An nginx web server
2. gwas_flask: The Flask web application running behind a gunicorn WSGI server (see ./deploy/flask.Dockerfile)
3. gwas_cron: A cron scheduler to generate new data daily (see ./deploy/cron.Dockerfile)

To launch the Docker containers from the command line:

cd .../gwasdiversitymonitor
docker-compose up -d --build

You can check the status of active docker containers using:

docker ps
docker stats

While the containers are running, you can view the app from your web browser by navigating to http://localhost:80.

To stop the containers, use:

docker-compose down

Versioning

This dashboard is currently at Version 1.0.0 (with an article conditionally accepted Nature Genetics). Please note: although the library logs data updates, it could be that additional dictionary based classifications are required with regards to the /data/support/dict_replacer_broad.tsv file. Please raise an issue in this repo to alert us of any necessary entries, or any suggestions which you may have in general, although we will monitor this over time.

License

This work is free. You can redistribute it and/or modify it under the terms of the MIT license, subject to the conditions regarding the data imposed by the EMBL-EBI license. The dashboard comes without any warranty, to the extent permitted by applicable law.

Acknowledgements

We are grateful to the extensive help provided by Global Initiative (and in particular to Jamie, Nynke, Veatriki, Alex, Lea and Quentin). Additional help and specific comments came from Ian Knowles, Yi Liu, Jiani Yan, Molly Przeworski, Ben Domingue, Sam Trejo and the Sociogenome group more generally.

Future Versions

1. Additional tabs regarding funders, network analysis, and so forth.