CartoDB / observatory-extension

BSD 3-Clause "New" or "Revised" License
6 stars 4 forks source link

Deal with not valid and NULL geometries in the input geomvals #291

Closed ethervoid closed 7 years ago

ethervoid commented 7 years ago

Related #290, #844

If we pass invalid or NULL geometries in the geomvals this could lead strange behavior like think that a points dataset is made of polygons

javitonino commented 7 years ago

I did some experiments with this, adding either ST_IsValid or ST_MakeValid to the GetData function. Some findings over the dataset in https://github.com/CartoDB/bigmetadata/issues/222 (5k polygons, 60 invalid ones, average 40 points/polygon, takes around 6 seconds to process):

I'm not too sure on how to go with this one. If we are optimizing for speed, we could keep the extension code and only add this check to the camshaft node. Maybe add some code in camshaft so when an exception is thrown by GetData, this error is bubbled up to the user, instead of silently setting the entire column to NULL.

noguerol commented 7 years ago

Closing this in favor of https://github.com/CartoDB/bigmetadata/issues/233 so we can better track it