search

willtemperley / osm-hadoop

Tools for extracting geospatial data from OpenStreetMap
Apache License 2.0
9 stars 1 forks source link

Spark rasterization hangs #1

Open willtemperley opened 8 years ago

willtemperley commented 8 years ago

The WayRasterizer in (mapreduce) is fast, but porting to Spark has been problematic - it hangs for no apparent reason. Also need to find a better way to broadcast the WKTReader.

https://github.com/willtemperley/osm-hadoop/blob/master/src/main/scala/org/roadlessforest/osm/SparkRasterizer.scala

willtemperley commented 8 years ago

P.S. the input will be WayWritables which hold a WKT string and a value to rasterize (e.g. highway values).