Open geertoff opened 2 months ago
Hey! Try changing the argument raw_data to df = scraper.run(raw_data=True, save=True, filepath="test2.csv")
That returned a non-empty data frame and worked for me.
Hi @zencodess, this doesn't work, unfortunately. Everything is NAs, except date_list
, city
, and log_id
I've got the same problem. Is there already a sollution?
When i use the following script for sold houses it works perfect, but for houses which are for sale it doesn't work, then i get an empty dataframe.
from funda_scraper import FundaScraper
if name == "main": scraper = FundaScraper( area="gemeente (naam gemeente)", want_to="buy", find_past=True, page_start=1, n_pages=10, ) df = scraper.run(raw_data=False, save=True, filepath="test.csv") print(df.head())
Seems like the CSS selectors have been changed/obfuscated on the funda website...
They started protecting from scraping them via requests. I'm not sure what can be a solution here rather than something like Selenium but that would be much longer.
They started protecting from scraping them via requests. I'm not sure what can be a solution here rather than something like Selenium but that would be much longer.
https://github.com/whchien/funda-scraper/issues/40 issue was solved in the past. But these pre-beta pages are not available anymore. CSS has just been changed.
I don't think Selenium would be necessary, we would just need more complicated HTML/CSS selectors.
Hi,
I've tried running the scraper using the script listed in the documentation on Windows, however I get a dataframe with no records... I've cloned the repository, installed a conda environment using the
requirements.txt
. and updatedurrlib3
after I got theModuleNotfoundError
. It looks like it does extract information from the individual links, but not saving it to the dataframe. See console output below.Script that is run:
Is there something I forgot?