Create an API

[lingyielia]

A "stretch" goal for this project.

Create an API to access the LA cross walk file.

• Option 1: Use Carto.com

  • Upload our final CSV file to Carto and export it as API.

• Option 2: Use dirt simple postgis http api

  • Host using AWS

[vr00n]

@lingyielia FYI i updated the above for clarity.

[gauravcusp]

I used the result file and converted it to json file . After that i created a local json-server to host the api and run it on Postman GUI.

I've hosted the file on AWS as well but working on api file for that.

@vr00n We will have slightly more clarity on API after your AWS session i believe. Until then can you let us know if this what was required.

[vr00n]

Fantastic work @gauravcusp !

[gauravcusp]

@vr00n I attempted to work with Python and Flask to create and API and host data on local server. I managed to get the data on server but since the data is enormous, its really slow. I've placed the files here along with README to re-create the steps. Also, this is the result I got from API: (Please disregard the first block as I intend to remove it, but the file size is making it difficult to edit)

@lingyielia @xd515

This is the what it looks like when the python script runs. It will capture all the requests and the query that you pass:

[vr00n]

AWESOME work @gauravcusp

Curious does this handle spatial querying like : https://varun-cusp2.carto.com/api/v2/sql?q=SELECT%20*%20FROM%20master_crosswalk_nad83%20where%20ST_Contains(the_geom,%20ST_GeomFromText(%27POINT(-73.952686%2040.775745)%27,4326)) ?

Lets add this to tomorrow's meeting.

[gauravcusp]

@vr00n The crosswalk documents didn't have geography column attached so the query won't work. Also since I didn't connect the script with a database, query won't return any result. I stored the json file on AWS and extracted data using the link. If needed, I can work on script that uses database instead of a json file directly.

[gauravcusp]

Hi @vr00n I've prepared the diagram for API here. Please go through it and let me know if my understanding is correct. @lingyielia @xd515

[vr00n]

Great work @gauravcusp. Few comments:

1. Number the arrows to create a sequence of actions
2. How does the Lambda "populate" the database? The data, once created, stays in the DB. This is just a simple select query right?

[gauravcusp]

@vr00n What i was thinking was using the switch statement for every city we use. If the user location is in NY, the script would extract the data from our NY file and create a table with that data. Query would be performed after that on the populated data (something like CREATE TABLE IF NOT EXISTS ABC).

Another approach could be, prepoluate data while setting up database and then.perform select queries on them based on the location.

I hope I'm not lacking in the understanding of the workflow. I'll number the arrows but basically the process starts with UI and ends there as well.

[vr00n]

Noted. For the time being, lets keep it simple.

Load the LA + NYC data to PostGres as 2 separate tables.

If:
User queries a location from NYC - then select from NYC table 
elseif :
User queries a location from LA - then select from LA table 

[gauravcusp]

Updated

[patwater]

This looks super cool! Are you all able to present on Friday?

[gauravcusp]

Hey @patwater do you mean present the API on Friday? Because we're still in early stages of its development.

[lingyielia]

LA data loaded on the AWS RDS now. Here shows how we did it.

[patwater]

@gauravcusp idea with Friday is just a 10 minute informal presentation to show progress. So could talk about what's involved with generating a geo-cross-walk, the data you integrated and the API could be a final bit about what's to come

[gauravcusp]

@patwater sure we can do that. Let us know about the meeting on Friday (time and schedule).

[patwater]

10 AM to noon PST = 1 PM -3 PM EST. Did you get the cal invite?

[lingyielia]

@patwater , I haven't received the call invite.

[patwater]

Could you please send me an email? I'll add you to the cal invite

[gauravcusp]

I got one for Saturday's meeting. I hope that's what you're talking about.

[patwater]

@gauravcusp it's Friday not Saturday. There's a separate Saturday workshop on dash as a data viz tool

[gauravcusp]

@patwater In that case, please send me the invite too. My id is bhardwaj@nyu.edu

[lingyielia]

@vr00n @gauravcusp @xd515 @patwater Guys, check this out! A simple LA County Geocrosswalk App

[gauravcusp]

Additional Note: We have hosted the app on heroku since there were some complications with AWS.

[gauravcusp]

@vr00n Can we have the shapefile for NYC as well? We can incorporate that with our code and make it public.

[vr00n]

@gauravcusp uploaded zipped shapefile https://github.com/argo-marketplace/LA-LocalGeo-CrossWalk/blob/master/NYC_master_crosswalk_nad83.zip

[vr00n]

@lingyielia Tested and verified looking AWESOME!!!

https://calm-stream-52725.herokuapp.com/results?lonSearch=-118.520259&latSearch=34.215713

[gauravcusp]

I asked some of my friends for their feedback and I think they can be incorporated in the future. I'll create another issue for that and we can probably work on it sometime in future.

[lingyielia]

@vr00n @gauravcusp Cool! I will update the README soon.

[vr00n]

Thanks @gauravcusp - Can you please update the README with heroku deployment step-by-step instructions please ?

Thanks!

[lingyielia]

@vr00n This readme explained how I build this app. If anything is not clear please let me know.

[lingyielia]

APP deployed here, doc here.