NPR's Book Concierge: Our Guide To 2013's Great Reads.
This code is open source under the MIT license. See LICENSE for complete details.
The following things are assumed to be true in this documentation.
~/Dropbox (the default location) and you have the nprapps folder synchronized.For more details on the technology stack used with the app-template, see our development environment blog post.
The project contains the following folders and important files:
confs -- Server configuration files for nginx and uwsgi. Edit the templates then fab <ENV> render_confs, don't edit anything in confs/rendered directly.data -- Data files, such as those used to generate HTML.etc -- Miscellaneous scripts and metadata for project bootstrapping.jst -- Javascript (Underscore.js) templates.less -- LESS files, will be compiled to CSS and concatenated for deployment.templates -- HTML (Jinja2) templates, to be compiled locally.tests -- Python unit tests.www -- Static and compiled assets to be deployed. (a.k.a. "the output")www/assets -- A symlink to a Dropbox folder containing binary assets (images, audio).www/live-data -- "Live" data deployed to S3 via cron jobs or other mechanisms. (Not deployed with the rest of the project.)www/test -- Javascript tests and supporting files.app.py -- A Flask app for rendering the project locally.app_config.py -- Global project configuration for scripts, deployment, etc.copytext.py -- Code supporting the Editing workflowcrontab -- Cron jobs to be installed as part of the project.fabfile.py -- Fabric commands automating setup and deployment.public_app.py -- A Flask app for running server-side code.render_utils.py -- Code supporting template rendering.requirements.txt -- Python requirements.Node.js is required for the static asset pipeline. If you don't already have it, get it like this:
brew install node
curl https://npmjs.org/install.sh | shThen bootstrap the project:
cd books13
mkvirtualenv --no-site-packages books13
pip install -r requirements.txt
fab bootstrapProject secrets should never be stored in app_config.py or anywhere else in the repository. They will be leaked to the client if you do. Instead, always store passwords, keys, etc. in environment variables and document that they are needed here in the README.
The secrets for this project are stored in DropBox in books13/baker_taylor_creds.txt; please add these to your ~/.bashrc or ~/.zshrc.
Any copyrighted or large binary assets (images, audio, video), should not be added to the Github repository, but rather to the folder in Dropbox corresponding to this project: ~/Dropbox/nprapps/assets/$NEW_PROJECT_NAME. This folder is symlinked to www/assets during the bootstrap process.
These assets will be deployed, but will not be committed to the repository. This is both make cloning the repository faster and also to make it easier to open source new projects.
This application requires Baker & Taylor credentials to download book covers. This should be set as environment variables books13_BAKER_TAYLOR_USERID and books13_BAKER_TAYLOR_PASSWORD. Once set, run fab load_images to download them to your dropbox folder.
A site can have any number of rendered templates (i.e. pages). Each will need a corresponding view. To create a new one:
templates directory. Ensure it extends _base.html.app.py. Decorate it with a route to the page name, i.e. @app.route('/filename.html').html and do not start with _ will automatically be rendered when you call fab render.A flask app is used to run the project locally. It will automatically recompile templates and assets on demand.
workon books13
fab update_copy
fab bootstrap
python app.pyVisit localhost:8000 in your browser.
This app uses a Google Spreadsheet for a simple key/value store that provides an editing workflow.
View the sample copy spreadsheet.
This document is specified in app_config with the variable COPY_GOOGLE_DOC_KEY. To use your own spreadsheet, change this value to reflect your document's key (found in the Google Docs URL after &key=).
A few things to note:
key, there is expected to be a column called value and rows will be accessed in templates as key/value pairsThis document is specified in app_config with the variable COPY_GOOGLE_DOC_KEY. To use your own spreadsheet, change this value to reflect your document's key (found in the Google Docs URL after &key=).
The app template is outfitted with a few fab utility functions that make pulling changes and updating your local data easy.
To update the latest document, simply run:
fab update_copyNote: update_copy runs automatically whenever fab render is called.
At the template level, Jinja maintains a COPY object that you can use to access your values in the templates. Using our example sheet, to use the byline key in templates/index.html:
{{ COPY.attribution.byline }}More generally, you can access anything defined in your Google Doc like so:
{{ COPY.sheet_name.key_name }}You may also access rows using iterators. In this case, the column headers of the spreadsheet become keys and the row cells values. For example:
{% for row in COPY.sheet_name %}
{{ row.column_one_header }}
{{ row.column_two_header }}
{% endfor %}With the project running, visit localhost:8000/test/SpecRunner.html.
Python unit tests are stored in the tests directory. Run them with fab tests.
Compile LESS to CSS, compile javascript templates to Javascript and minify all assets:
workon books13
fab render(This is done automatically whenever you deploy to S3.)
If you want to test the app once you've rendered it out, just use the Python webserver:
cd www
python -m SimpleHTTPServerfab staging master deployYou can deploy to EC2 for a variety of reasons. We cover two cases: Running a dynamic web application (public_app.py) and executing cron jobs (crontab).
Servers capable of running the app can be setup using our servers project.
For running a Web application:
app_config.py set DEPLOY_TO_SERVERS to True.app_config.py set DEPLOY_WEB_SERVICES to True.fab staging master setup_server to configure the server.fab staging master deploy to deploy the app.For running cron jobs:
app_config.py set DEPLOY_TO_SERVERS to True.app_config.py, set INSTALL_CRONTAB to Truefab staging master setup_server to configure the server.fab staging master deploy to deploy the app.You can configure your EC2 instance to both run Web services and execute cron jobs; just set both environment variables in the fabfile.
Cron jobs are defined in the file crontab. Each task should use the cron.sh shim to ensure the project's virtualenv is properly activated prior to execution. For example:
* * * * * ubuntu bash /home/ubuntu/apps/$PROJECT_NAME/repository/cron.sh fab $DEPLOYMENT_TARGET cron_testNote: In this example you will need to replace $PROJECT_NAME with your actual deployed project name.
To install your crontab set INSTALL_CRONTAB to True in app_config.py. Cron jobs will be automatically installed each time you deploy to EC2.
Web services are configured in the confs/ folder.
Running fab setup_server will deploy your confs if you have set DEPLOY_TO_SERVERS and DEPLOY_WEB_SERVICES both to True at the top of app_config.py.
To check that these files are being properly rendered, you can render them locally and see the results in the confs/rendered/ directory.
fab render_confsYou can also deploy the configuration files independently of the setup command by running:
fab deploy_confsSometimes it makes sense to run a fabric command on the server, for instance, when you need to render using a production database. You can do this with the fabcast fabric command. For example:
fab staging master fabcast:deployIf any of the commands you run themselves require executing on the server, the server will SSH into itself to run them.