cfpb / grasshopper

CFPB's streaming batch geocoder
Creative Commons Zero v1.0 Universal
37 stars 13 forks source link

Move test-harness's Elasticsearch setup from Docker Compose to local install #209

Closed hkeeler closed 8 years ago

hkeeler commented 8 years ago

Elasticsearch performance is very poor when running under our development Docker Machine VirtualBox setup. This is primarily due to VirtualBox's slow I/O, which is a reaaaally painful when pushing many gigs worth of address data at it, as we need to do for using the test-harness.

To get around this, let's move Elasticsearch out of Docker Compose, and run it locally instead. In initial testing, I've been able to load all points data in about 30 minutes, which beats several hours under the Docker setup.

This fix will also include the upgrade to Elasticsearch 2.2, making this dependent on PR #206.