search

whchien / funda-scraper

FundaScaper scrapes data from Funda, the Dutch housing website. You can find listings from house-buying or rental market, and historical data. 🏡
GNU General Public License v3.0
104 stars 48 forks source link

Change type bool args to store_true action #31

Closed leonlan closed 3 months ago

leonlan commented 6 months ago

Hi @whchien, thanks for building this scraper! I was using this recently from the CLI and I noticed that some args are not correctly parsed. For example, running the command from the README:

❯ python funda_scraper/scrape.py --area amsterdam --want_to rent --find_past False --page_start 1 --n_pages 3 --raw_data True

When printing args.find_past, this evaluates to True meaning that it unintentionally uses past data even though I expected recent data.

This is because some args are parsed as type=bool, which unfortunately often leads to a different behavior than expected. "False" is interpreted as string and that evaluates True in Python. On the other hand, inputs like '' and 0 will evaluate as False. The same problem occurs with the --raw_data and --save arguments.

One way to avoid this problem is to use action=store_true instead. I have made these changes along with a few small quality-of-life improvements.

utkuarslan5 commented 5 months ago

Thank you! Been looking for the source of the find_past=False bug for a while