search

nprapps / elections14

We're having an election party.
http://elections.npr.org
MIT License
11 stars 2 forks source link

readme

elections14

What is this?

TKTK: Describe elections14 here.

Assumptions

The following things are assumed to be true in this documentation.

For more details on the technology stack used with the app-template, see our development environment blog post.

What's in here?

The project contains the following folders and important files:

Bootstrap the project

Node.js is required for the static asset pipeline. If you don't already have it, get it like this:

brew install node
curl https://npmjs.org/install.sh | sh

Before bootstrapping, you will also need to add a new set of credentials to your environment variables. Ask somebody on the team.

Then bootstrap the project:

cd elections14
mkvirtualenv elections14
pip install -r requirements.txt
npm install
fab text
fab ap.init
fab data.bootstrap

Problems installing requirements? You may need to run the pip command as ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements.txt to work around an issue with OSX.

Problems with data.bootstrap? Run createdb to initialize your user database, and make sure you have the project environment variables in your .bash_profile.

Simulate election results

To apply random poll closing times, vote counts, race calls, etc. to default Associated Press data, run:

fab data.mock_results

Display the big board in the building

The Senate big board is available on IPTV on channel 151. The internal IP address for administration is 10.32.64.205.

Reset server and database

To reset a server and database, run:

fab staging master data.reset_server

Hide project secrets

Project secrets should never be stored in app_config.py or anywhere else in the repository. They will be leaked to the client if you do. Instead, always store passwords, keys, etc. in environment variables and document that they are needed here in the README.

Save media assets

Large media assets (images, videos, audio) are synced with an Amazon S3 bucket specified in app_config.ASSETS_S3_BUCKET in a folder with the name of the project. (This bucket should not be the same as any of your app_config.PRODUCTION_S3_BUCKETS or app_config.STAGING_S3_BUCKETS.) This allows everyone who works on the project to access these assets without storing them in the repo, giving us faster clone times and the ability to open source our work.

Syncing these assets requires running a couple different commands at the right times. When you create new assets or make changes to current assets that need to get uploaded to the server, run fab assets.sync. This will do a few things:

Unfortunantely, there is no automatic way to know when a file has been intentionally deleted from the server or your local directory. When you want to simultaneously remove a file from the server and your local environment (i.e. it is not needed in the project any longer), run fab assets.rm:"www/assets/file_name_here.jpg"

Adding a page to the site

A site can have any number of rendered pages, each with a corresponding template and view. To create a new one:

Run the project

A flask app is used to run the project locally. It will automatically recompile templates and assets on demand.

workon $PROJECT_SLUG
python app.py

Visit localhost:8000 in your browser.

COPY editing

This app uses a Google Spreadsheet for a simple key/value store that provides an editing workflow.

View the sample copy spreadsheet.

This document is specified in app_config with the variable COPY_GOOGLE_DOC_KEY. To use your own spreadsheet, change this value to reflect your document's key (found in the Google Docs URL after &key=).

A few things to note:

The app template is outfitted with a few fab utility functions that make pulling changes and updating your local data easy.

To update the latest document, simply run:

fab copytext.update

Note: copytext.update runs automatically whenever fab render is called.

At the template level, Jinja maintains a COPY object that you can use to access your values in the templates. Using our example sheet, to use the byline key in templates/index.html:

{{ COPY.attribution.byline }}

More generally, you can access anything defined in your Google Doc like so:

{{ COPY.sheet_name.key_name }}

You may also access rows using iterators. In this case, the column headers of the spreadsheet become keys and the row cells values. For example:

{% for row in COPY.sheet_name %}
{{ row.column_one_header }}
{{ row.column_two_header }}
{% endfor %}

When naming keys in the COPY document, pleaseattempt to group them by common prefixes and order them by appearance on the page. For instance:

title
byline
about_header
about_body
about_url
download_label
download_url

Arbitrary Google Docs

Sometimes, our projects need to read data from a Google Doc that's not involved with the COPY rig. In this case, we've got a class for you to download and parse an arbitrary Google Doc to a CSV.

This solution will download the uncached version of the document, unlike those methods which use the "publish to the Web" functionality baked into Google Docs. Published versions can take up to 15 minutes up update!

First, export a valid Google username (email address) and password to your environment.

export APPS_GOOGLE_EMAIL=foo@gmail.com
export APPS_GOOGLE_PASS=MyPaSsW0rd1!

Then, you can load up the GoogleDoc class in etc/gdocs.py to handle the task of authenticating and downloading your Google Doc.

Here's an example of what you might do:

import csv

from etc.gdoc import GoogleDoc

def read_my_google_doc():
    doc = {}
    doc['key'] = '0ArVJ2rZZnZpDdEFxUlY5eDBDN1NCSG55ZXNvTnlyWnc'
    doc['gid'] = '4'
    doc['file_format'] = 'csv'
    doc['file_name'] = 'gdoc_%s.%s' % (doc['key'], doc['file_format'])

    g = GoogleDoc(**doc)
    g.get_auth()
    g.get_document()

    with open('data/%s' % doc['file_name'], 'wb') as readfile:
        csv_file = list(csv.DictReader(readfile))

    for line_number, row in enumerate(csv_file):
        print line_number, row

read_my_google_doc()

Google documents will be downloaded to data/gdoc.csv by default.

You can pass the class many keyword arguments if you'd like; here's what you can change:

See etc/gdocs.py for more documentation.

Run Python tests

Python unit tests are stored in the tests directory. Run them with fab tests.

Run Javascript tests

With the project running, visit localhost:8000/test/SpecRunner.html.

Compile static assets

Compile LESS to CSS, compile javascript templates to Javascript and minify all assets:

workon elections14
fab render

(This is done automatically whenever you deploy to S3.)

Test the rendered app

If you want to test the app once you've rendered it out, just use the Python webserver:

cd www
python -m SimpleHTTPServer

Deploy to S3

fab staging master deploy

Deploy to EC2

You can deploy to EC2 for a variety of reasons. We cover two cases: Running a dynamic web application (admin_app.py) and executing cron jobs (crontab).

Servers capable of running the app can be setup using our servers project.

For running a Web application:

For running cron jobs:

You can configure your EC2 instance to both run Web services and execute cron jobs; just set both environment variables in the fabfile.

Bootstrapping the server

fab staging master servers.fabcast:"data.bootstrap"
fab staging master server.fabcast:"data.mock_results"

Install cron jobs

Cron jobs are defined in the file crontab. Each task should use the cron.sh shim to ensure the project's virtualenv is properly activated prior to execution. For example:

* * * * * ubuntu bash /home/ubuntu/apps/elections14/repository/cron.sh fab $DEPLOYMENT_TARGET cron_jobs.test

To install your crontab set INSTALL_CRONTAB to True in app_config.py. Cron jobs will be automatically installed each time you deploy to EC2.

The cron jobs themselves should be defined in fabfile/cron_jobs.py whenever possible.

Install web services

Web services are configured in the confs/ folder.

Running fab servers.setup will deploy your confs if you have set DEPLOY_TO_SERVERS and DEPLOY_WEB_SERVICES both to True at the top of app_config.py.

To check that these files are being properly rendered, you can render them locally and see the results in the confs/rendered/ directory.

fab servers.render_confs

You can also deploy only configuration files by running (normally this is invoked by deploy):

fab servers.deploy_confs

Run a remote fab command

Sometimes it makes sense to run a fabric command on the server, for instance, when you need to render using a production database. You can do this with the fabcast fabric command. For example:

fab staging master servers.fabcast:deploy

If any of the commands you run themselves require executing on the server, the server will SSH into itself to run them.

Analytics

The Google Analytics events tracked in this application are:

Category Action Label Value Custom 1 Custom 2
elections14 chromecast-initiated
elections14 chromecast-stopped
elections14 fullscreen
elections14 state-selected state
elections14 audio-toggle
elections14 audio-fail
elections14 slide-link-click
elections14 next-slide-click
elections14 prev-slide-click
elections14 keyboard-nav
elections14 facebook share-discuss
elections14 tweet share-discuss
elections14 mobile-controls