Load CIP Lines API into Postgres using Django

hackoregon / transportation-backend

Transportation backend repo - 2016 and 2017

8 stars 7 forks source link

Load CIP Lines API into Postgres using Django #7

Closed sanjuroj closed 7 years ago

sanjuroj commented 7 years ago

Create a django loader for the CIP Lines API. See the django importer for the CIP Points API for the approach to loading that has been taken so far.

Documentation on APIs can is in this google docs folder.

For now, load the data just once. We will eventually want to regularly import new and updated data from the city APIs, but that will require extra work to ensure we merge the new data with the existing data correctly. For now, we just need a data set in the database so we can create a demo frontend.

sanjuroj commented 7 years ago

While you're working on this, you may want take a look at all six of the point, line, poly APIs. They may be similar enough to each other that they can be handled by a single abstracted importer, with everything going into a Project model.

sanjuroj commented 7 years ago

Your updates look good! Now how is this for an idea: the only thing in this loader that is specific to an API is the start and end dates. So what if we made a constant that specified the field names for start date and end date for each city API we want to process. Then we could loop through any number of Line APIs from the city and push them into our main Line model. Maybe we can create a separate ticket for that :-)