search

UrbanCCD-UChicago / plenario2

The Premier Open Data Discovery Platform
http://plenar.io/
Other
12 stars 2 forks source link

Toward a sane internal API and Socrata native ingests #460

Closed vforgione closed 5 years ago

vforgione commented 5 years ago

The internal API was getting really nasty -- we had a bunch of one off functions that clashed in arity (positional arguments, matches, guards, options ...).

The web application was also a disaster -- originally I thought it would make it easier to keep the web, admin and API separate in subapps, but that ended up making things that much more difficult.

Then that leaves the elephant in the room: Socrata. We've always relied on them and all of their awful decisions. The changes here in remove some of the terrible things about Socrata integration and makes ingesting their data sets a little cleaner.

Breaking Changes:

Closes #235 Closes #340 Closes #360 Closes #361

vforgione commented 5 years ago

So, before you say that this rips out a bunch of schemas and functionality and that kind of is a step backward -- I'll meet you half way and say a lot of the old code that was removed was pointless cruft that added very little, to basically nothing and maybe even confusing, functionality to the overall aim of the project.

I think the only thing that was potentially useful that I dropped was support for shapefiles. I'm not convinced that it's necessary functionality.

This focuses on doing a few things, and doing them well:

vforgione commented 5 years ago

Most recent commit adds back bbox and time range filtering and pagination to the detail endpoint. Bbox and time range are now generic and don't rely on knowing exact field names ahead of time.