search

zooniverse / classroom-maps-api

Mapping API for the https://classroom.zooniverse.org service. Built using https://datasette.io/ to provide a JSON API over SQLite databases with custom project CSV data.
MIT License
2 stars 0 forks source link

Allow GeoJson renderer to handle Carto's encrypted 'the_geom' data #2

Closed shaunanoordin closed 3 years ago

shaunanoordin commented 3 years ago

PR Overview

Context: for reasons that evade me, the old Carto database sometimes encodes its geographical data into a "the_geom" field. This field contains the GeoJSON feature, but encoded as a string in the well known binary format.

This PR expands the GeoJSON renderer to read the old "the_geom" field and translate it into a proper JSON-ified object that can be read by some JavaScript.

Status

WIP - I need to figure out how to install the plpygis Python library to Docker, since I'm getting ModuleNotFoundError: No module named 'plpygis' when running docker-compose up.

shaunanoordin commented 3 years ago

Oh, duh! Dockerfile: RUN pip install csvs-to-sqlite sqlite-utils geojson plpygis . I was searching for the string 'python' throughout the project to find the pattern when I should have been looking for pip.

shaunanoordin commented 3 years ago

New issue: the If conditions aren't triggering.

if hasattr(row, 'longitude') and hasattr(row, 'latitude'):  # always returns false and false

if 'longitude' in row and 'latitude' in row:  # always returns false and false

Gotta dig into the Python docs for this. Maybe the Row object is negating the standard member checking methods.

shaunanoordin commented 3 years ago

PR Update

This PR is now ready for review! My only concern now is that the fix to the 'in' check was to change if 'the_geom' in row: to if 'the_geom' in row.keys():... which works but I suspect may be more resource-intensive due to calling a function.

shaunanoordin commented 3 years ago

To test this, try:

Both should return a feature collection with various data points. Data is showing up fine on local Zooniverse Classrooms instance.

shaunanoordin commented 3 years ago

👍