search

whchien / funda-scraper

FundaScaper scrapes data from Funda, the Dutch housing website. You can find listings from house-buying or rental market, and historical data. 🏡
GNU General Public License v3.0
104 stars 48 forks source link

Multiple neighbourhoods #13

Closed NelisV closed 3 months ago

NelisV commented 1 year ago

When i'm browsing funda manually i have a custom query based on a list of multiple neighborhoods; all based on the 'CBS Wijken en Buurten' dataset.

Would it be possible to let the input of area be a list with neighbourhoods?

Much appreciated! your script is a lifesaver!

NelisV commented 1 year ago

Example of a combined URL: "https://www.funda.nl/zoeken/koop?selected_area=%5B%22delft%2Fwippolder-noord%22%2C%22delft%2Fcentrum-oost%22%2C%22delft%2Fcentrum-zuidoost%22%2C%22delft%2Fcentrum%22%2C%22delft%2Fcentrum-west%22%2C%22delft%2Fcentrum-zuidwest%22%2C%22den-haag%2Fkortenbos%22%2C%22den-haag%2Fzuidwal%22%2C%22den-haag%2Fvoorhout%22%2C%22den-haag%2Fwillemspark%22%2C%22den-haag%2Fzeeheldenkwartier%22%2C%22den-haag%2Farchipelbuurt%22%2C%22den-haag%2Fstatenkwartier%22%2C%22leiden%2Facademiewijk%22%2C%22leiden%2Fpieterswijk%22%2C%22leiden%2Fde-camp%22%2C%22leiden%2Fmolenbuurt%22%2C%22leiden%2Fnoordvest%22%2C%22leiden%2Fhavenwijk-noord%22%2C%22leiden%2Fmarewijk%22%2C%22leiden%2Fpancras-west%22%2C%22leiden%2Fpancras-oost%22%2C%22leiden%2Fhavenwijk-zuid%22%2C%22leiden%2Flevendaal-oost%22%2C%22leiden%2Flevendaal-west%22%2C%22leiden%2Fd-oude-morsch%22%5D&sort=%22floor_area_down%22&price=%22-300000%22"

As neighbourhoods in different cities can have the same name, the city has to be added to make it specific.

Riik commented 11 months ago

Did you try what happens if you add everything after selected_area= up untill the next parameter / & in the URL? From the code it looks like the "area" parameter is directly inserted there, so I think it should work to just use that very long string directly. See here in code: https://github.com/whchien/funda-scraper/blob/5e592f47e0500a0199c1bd7c02ac27cb26d0d675/funda_scraper/scrape.py#L145

whchien commented 3 months ago

@NelisV @Riik Hi, sorry for the late reply. You can indeed pass multiple neighborhoods to construct the main URL, but I believe this approach is more error-prone. I suggest instantiating several FundaScraper objects and passing different area arguments in a for loop to achieve the same outcome.