add initial spatial-index.preprocessor implementation

[artemp]

@GretaCB - my first cut and it needs some more work

[springmeyer]

I got the syntax checks working. Next steps:

• [x] Add unit tests
• [x] @GretaCB review all the code
• [x] Currently only generating indexes for files > 50MB
  • [x] should we reduce size?
  • [ ] is there a better place to have this threshold/number come from?
• [ ] Should we attempt to process multiple files as in shp-index.preprocessor.js ?

[springmeyer]

travis is failing on one test which we need to look into:

not ok 113 shp-index projected to spherical mercator
  ---
    operator: ok
    expected: true
    actual:   false
    at: DatasetLayers.defaultForEach [as forEach] (/home/travis/build/mapbox/preprocessorcerer/node_modules/gdal/lib/gdal.js:80:7)

This is failing in unrelated code - so it may have to do with the node-mapnik upgrade.

[springmeyer]

This is failing in unrelated code - so it may have to do with the node-mapnik upgrade.

The failing test has only to do directly with node-gdal: https://github.com/mapbox/preprocessorcerer/blob/mapnik-index/test/end2end.test.js#L77. The node-mapnik version upgraded GDAL so we may have an ABI incompatibility sneaking in. Next thing to try will be upgrading to node-gdal v0.7.2 which uses gdal 2.0.1 like node-mapnik 3.4.10 which also bumpt proj4: https://github.com/mapnik/node-mapnik/blob/mp-union/CHANGELOG.md#3410

[GretaCB]

Also @artemp , let me know if you need any help with tests. Should be pretty similar to shp-index tests.

[springmeyer]

@artemp @GretaCB - I added tests to help push this forward. I also bumped the indexable size down to 10 MB. I think this is ready to stage with some large files.

[artemp]

@springmeyer - thanks for the tests! I'm personally think 10MB is a bit on a small side but lets try.

[springmeyer]

@GretaCB just merged latest master into the mapnik-index - hoping that will fix the failing test which was due to #47

[GretaCB]

@artemp Do you have any particular GeoJSON files you've been using to test indexing in Mapnik?