When running in local mode, Countries.all being a def results in countries.geojson being parsed at the beginning of each task execution (and concurrently to the tune of the number of cores available). This allows parsing to occur once over the lifetime of the job.
This could use some general cleanup and also needs to be tested on a proper Spark cluster before being merged.
When running in local mode,
Countries.all
being adef
results incountries.geojson
being parsed at the beginning of each task execution (and concurrently to the tune of the number of cores available). This allows parsing to occur once over the lifetime of the job.This could use some general cleanup and also needs to be tested on a proper Spark cluster before being merged.
Refs #77