Housing/Rent Prices - Githubissues

Data4Democracy / usa-dashboard

A dashboard of key metrics for the USA

69 stars 27 forks source link

Housing/Rent Prices #22

Open seanjtaylor opened 7 years ago

seanjtaylor commented 7 years ago

What's the cost of housing? Are people paying more for housing or rent than they were last year?

Ideas:

craigslist posts
Zillow/Redfin
financial markets data
homelessness reports

josephpd3 commented 7 years ago

I have experience building scrapers and can help with a little bit of direction! I'd love to explore and learn more about the housing/rent market. I think it would be really interesting to be able to control for region, demographic, etc with respect to some of these data sources.

I'll take a look around for sources leading up to and through the weekend. If y'all need someone for a few hours of grunt-work on this during the hackathon, feel free to ping me in slack! (Same username)

josephpd3 commented 7 years ago

Update: I looked at the APIs/data for Redfin and Zillow:

Zillow's API has super restrictive terms of use. You have to follow strict branding guidelines, can't retain any data, and are limited on API calls per day (1000).
Redfin's data is really permitting and really cool. As long as you cite that you got the data from them, you can use it however you like. The demos on the linked page are really awesome and have the same approach to the data that we seem to desire here, though they don't have an open API to use like Zillow does.
- Redfin, however, also encapsulates their "reports" in Tableau format, so getting a clean CSV out of it may or may not be a hassle.

I believe a scraper could feasibly be done for Zillow/Redfin. I'll keep looking at the rest!

ckmarcelus1 commented 7 years ago

@seanjtaylor @josephpd3. Some of us (@sya, @alemosie, @dfm) are working through (craigslist, census data & Zillow's dataset in Data.World) to aggregate & clean up New York. https://www.zillow.com/research/data/ https://www.census.gov/housing/nychvs/data/2014/ser2a.html

Main fields that we are working on is: -Housing Sale Price -Housing Rental Price -Price/Square Footage -Zip Code -Address -Bedroom Number -Borough -Date of Posting -Neighborhood vacancy rate

josephpd3 commented 7 years ago

@ckmarcelus1 Awesome! I'm going to be available later on tonight to help some more, and I'd love to help dig through this if there is still digging to be done by then. Who should I PM on Slack when I'm available later?

ckmarcelus1 commented 7 years ago

@josephpd3 Hey, feel free to PM me and yeah we found some great datasets on Zillow's data site. They need some fair share of cleaning but pretty comprehensive and could be very valuable.

https://www.zillow.com/research/data/