A Flask app for managing the import and packaging of data sets for Open Humans users.
This app is currently designed to run on Heroku, using a web and worker dyno
whose configurations are specified in /Procfile
.
For local development, running this app with foreman
is strongly recommended,
as well as a \.env
file containing environment variable values (see
env.example
).
Running this site requires a PostgreSQL database (even for local development).
sudo su - postgres
createdb mydb
createuser -P jdoe
psql
GRANT ALL PRIVILEGES ON DATABASE mydb TO jdoe;
ALTER USER jdoe CREATEDB;
\q
exit
Specify the Postgres connection string in your .env
file, e.g.:
DATABASE_URL="postgres://jdoe:pa55wd@127.0.0.1/mydb"
Create the database tables like so:
$ foreman run python
>>> from utilities import init_db
>>> db = init_db()
>>> db.create_all()
The requests-respectful package requires a Redis server.
To set this up in Ubuntu using apt-get:
sudo apt-get install redis-server
To set this up in OSX using brew:
brew install redis
brew services start redis
Default configurations should work fine.
Putting these here for future reference, for understanding best practices in using S3 storage.
boto.get_bucket
performs a
"HEAD Bucket" operation to check that a bucket exists. This requires the
"s3:ListBucket" permission. It's also an option to run the command with
validate=False
, but this might produce confusing errors when a bucket doesn't
exist.arn:aws:s3:::my-bucket-name
) and object-level permissions
(using an ARN like arn:aws:s3:::my-bucket-name/*
).(Note: this takes a long time to run, and some tests demand nontrivial resources from other sites/repositories/etc. We should consider splitting up the tests, only running tests on updated aspects.)
Tests can be run using nosetests, e.g. nosetests data_retrieval/tests.py
.