CatalystCode / project-fortis-spark

A repository for all spark jobs running on fortis
MIT License
9 stars 4 forks source link

Ensure that place centroids are set in Cassandra #125

Closed c-w closed 7 years ago

c-w commented 7 years ago

This requires two pieces:

1) A feature recently was added to the featureService that enables the return of the centroid of a place using ST_Centroid in PostGIS. You can find a sample featureService request that includes place centroids in the response at https://aka.ms/b5ldsw. This feature is now integrated in the Spark pipeline so that we augment all detected locations with their WOF ids and their centroids.

2) The Cassandra aggregation logic and downstream GraphQL services assume that the centroid of places are always set. As such, we shouldn't represent the place coordinates in Spark as Optionals.