tsdataclinic / scout

Scout is a data discovery tool to explore open data portals worldwide.
https://scout.tsdataclinic.com
Apache License 2.0
33 stars 12 forks source link

[FEATURE] parse lat/long fields with more than one typo #366

Open NickKauf opened 1 year ago

NickKauf commented 1 year ago

NOTE: consider posting this feature request as a Discussion instead of an Issue

Is your feature request related to a problem? Please describe. trying to visualize this data set on a map, but the lat long is nested within parentheses i.e "POLYGON(lat, long)"

https://data.cityofnewyork.us/Recreation/NYC-Parks-Pools/y5rm-wagw)

Describe the solution you'd like so I'd like to be able to parse it out to view on the map

Describe alternatives you've considered manually parsing or writing a rule to pull out the lat long

Additional context image

jps327 commented 1 year ago

This is a great idea and one we should definitely prioritize on our end. Thanks for the heads up! We'll add some more ways to parse lat/long coordinates to improve the automatic detection but will also add some UI options for you to manually specify which are the lat/long columns. I'll update this issue once we kick off work on this and push the updates

jps327 commented 1 year ago

I do want to clarify though that this wouldn't solve the exact issue you're facing with this particular dataset. This dataset (NYC-Parks-Pools) doesn't have any lat/long column at all. The column in your screenshot is actually a polygon, i.e. each row is actually a list of lat/longs:

Screenshot 2023-03-18 at 2 55 33 PM

So each row is actually not a "point", each row is a region (i.e. a "polygon" - each row actually contains the points that map the entire perimeter of each pool).

We could consider adding a toggle to the map to change the feature rendering from points to polygons, and then you can specify which column contains the list of coordinates for each polygon. However this will take a longer to build though because this feature was built with points in mind, not polygons!

hydrosquall commented 1 year ago

In this case, each of the polygons is quite small (it's the shape of a pool), which when viewed from sufficient distance, might render the polygon too small to actually see (e.g. if you're at a level where you can actually see the shape of a borough). I suspect most of these pools will just look like dots once you're zoomed out.

In advance of offering polygon rendering support, perhaps it could simplify to offer the ability to reduce a polygon to a point (by finding its center ), and graphing the center points instead.

jps327 commented 1 year ago

@NickKauf just wanted to give an update that we've started work on this. We should have some more updates next week and hopefully push those changes to production