thinkingmachines / linksight-2018

LinkSight is a web app for applying the Philippine Standard Geographic Code to messy and misspelled barangay, municipality, city, and province names.
https://linksight.thinkingmachin.es
GNU General Public License v3.0
11 stars 1 forks source link

LinkSight

Build Status Coverage Status

This web app takes a list of barangay, municipality, city, and province names and looks up their closest matches in the Philippine Standard Geographic Code (PSGC). It's useful if you're trying to standardize location names for merging location datasets from diverse sources.

How it works

  1. Upload a single CSV file that contains separate barangay, province, and city or municipality columns.
  2. Identify the columns that correspond to the correct administrative levels in the PSG code.
  3. For places with exact matches in the PSGC, LinkSight returns the precise PSG code. For places with more than one possible matching PSG code, use the app's graphical user interface to select the correct match.
  4. Export the results as a CSV.

Feedback

This app is in active development! Your feedback will help us improve it. If you have any suggestions for features, enhancements, or find any bugs, or just want to contribute, please send us an email at linksight@thinkingmachin.es.

Development

  1. Install pm2

    npm install -g pm2
  2. Install dependencies

    make
  3. Set configuration values

    cp .env{.template,}
    Config name Description
    APPROVED_EMAILS_ROW_KEY The row key for approved emails
    APPROVED_EMAILS_SHEET_ID Google Sheets ID of th list of approved emails
    APPROVED_EMAILS_SHEET_RANGE The sheet range for approved emails
    DATABASE_URL See DJ-Database-URL
    DEBUG Toggles running the server in debug mode
    EMAIL_HOST_PASSWORD Password for above
    EMAIL_HOST_USER User for sending email
    EMAIL_PORT Port to use when sending email
    FLOWER_OAUTH2_REDIRECT_URI Redirect url for Flower auth
    GOOGLE_APPLICATION_CREDENTIALS Path to the Google Cloud Service Account key file used to retrieve the list of approved emails
    GOOGLE_OAUTH2_KEY Key for Google OAuth2
    GOOGLE_OAUTH2_SECRET Secret for Google OAuth2
    SECRET_KEY String used to provide cryptographic signing of sessions, etc.

    If you are a member of the official LinkSight development team, you can ask for access to the template with secrets filled in.

  4. Install postgresql and run the following:

    createdb linksight

    Make sure that you can connect using the DATABASE_URL you used in .env.

  5. Apply migrations:

    venv/bin/python manage.py migrate
  6. Start services:

    pm2 start
  7. Go to http://localhost:3000/

Testing

venv/bin/python manage.py test

You can run specific subsets of tests like so.

Deployment

NOTE: Commands below must be executed within the ./deploy directory.

Setup instances

  1. Create instances with terraform:

    make instance

    Note the IP shown in the output.

  2. Configure instances from the inventory file:

    cp hosts.yml{.template,}
  3. Setup instances:

    make setup-instances

Staging

cd deploy && ENV=staging HOST=linksight-stg.thinkingmachin.es make deploy

Production

cd deploy && ENV=production HOST=linksight.thinkingmachin.es make deploy

PSGC

The PSGC reference file is maintained in a separate repo and has been added to this repo as a git subtree. To pull latest changes, run:

git subtree pull --prefix data/psgc \
    https://github.com/thinkingmachines/psgc.git \
    master --squash