search

grantwilliams / wg-gesucht-crawler-cli

Python web crawler / scraper for WG-Gesucht. Crawls the WG-Gesucht site for new apartment listings and send a message to the poster, based off your saved filters and saved text
MIT License
76 stars 28 forks source link

Error while logging in #1

Closed matthlavacka closed 7 years ago

matthlavacka commented 7 years ago

At first, let me thank you for making this! Unfortunately, I'm having problems logging in.

This is an error message I receive when I try to login.

mhlavackas-MacBook-Air-2:wg-gesucht-crawler mhlavacka$ python main.py Exception in thread Thread-1: Traceback (most recent call last): File "//anaconda/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "//anaconda/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/Users/mhlavacka/Documents/wg-gesucht-crawler/wg_gesucht.py", line 28, in check_wg_credentials login_form.find("input", {"id": "email_user"})['value'] = login_info['email'] TypeError: 'NoneType' object does not support item assignment

grantwilliams commented 7 years ago

It seems wg-gesucht has changed their login form, making it not possible to use mechanicalsoup anymore. Post requests to the login form URL just get a 503 status code error. I have changed it now to use Selenium instead for the browser automation, you can see the README on how to set that up.