DistrictDataLabs / cultivar

Multidimensional data explorer and visualization tool.
http://trinket.districtdatalabs.com
Apache License 2.0
52 stars 18 forks source link

Closes #90. Update dev files #91

Closed lauralorenz closed 7 years ago

lauralorenz commented 7 years ago

This PR updates the docker-based and vagrant-based development scripts, configuration, and READMEs to reflect the current state of cultivar on develop. This PR falls just short of making the Vagrant set up full-service since it doesn't set up the messaging backend, but at least now it has a site.

How to test

You might want to start TOTALLY from scratch (re: Vagrant install, Docker install) but if not that far back, at least get rid of your old vagrant box and docker images just to be safe.

Follow the README sections from this PR on the dev environment with

  1. Docker
  2. Vagrant and confirm for each in turn that the site really goes up where the README says it will

You can also smoke test the messaging backend for the docker configuration by triggering a background tasks via uploading a data set and follow logs for celery (and rabbitmq if you like) to make sure that background tasks are being executed by workers to prove out the messaging backend

NOTE: I considered setting up an emailing backend for dev not worth fixing from a docker perspective (and it's already non existent from the Vagrant perspective) since for dev purposes we have no email configuration necessary at this point (no email logging and we don't appear to use it elsewhere in the project).

Implementation notes/notable things

To fix the Vagrant setup:

  1. This repo is not python 2.7 safe anymore as dicussed in #90, but the default python executable for the base vagrant box we're using (ubuntu/trusty32) points to python 2.7. So I switched the default by having the virtualenvironment the Vagrantfile provisions use the python3 executable (which is really python 3.4.1 in this ubuntu version, but we can deal with upgrades later if we want).

To fix the Docker setup:

  1. There was legacy stuff related to redis instead of rabbitmq as the backend, so switched it all over to rabbitmq which is what the django settings imply.
  2. Config management was off so I created a separate sample env file that handles the docker case, following the same env file antipattern as bare metal or vagrant dev installs
  3. Took out some irrelevant/configuration-complicating pieces from entrypoints (or entire entrypoints altogether) from the django and celery service containers, and added a wait loop so the django service will wait for postgres to allow connections since runserver won't exit or otherwise kick runserver again on a connection refused error while the postgres container is still starting up.
  4. Dropped configuring an smpt server since I decided the email backend was irrelevant for dev purposes right now

Overall fixes:

  1. There was a duplicate requirement in one of the inherited requirement files that pip was freaking out on for celery installs
  2. The README directions for adding null values to .env files were thinking pythony (ENV_VAR=None) instead of bashy (ENV_VAR=1) which is not ze same

Out of scope

  1. I didn't touch the 'prod' docker setup (docker-compose.yml)
  2. By my eye there is some complexity in the docker configuration that could be removed/streamlined, but only really touched the configuration-complexity introducing entrypoints for the celery and django service this round
  3. I didn't add any more db seeding/migration seeding then was already present
lauralorenz commented 7 years ago

@bahadasx if you want to take a look at this and see if the dev instructions work for you, I think this would be good to merge before we get anyone else into the codebase. I put testing instructions and an overview, but can go over it with you/anyone tomorrow(/today....) also cc @ojedatony1616 @bbengfort