Python web crawler / scraper for WG-Gesucht. Crawls the WG-Gesucht site for new apartment listings and send a message to the poster, based off your saved filters and saved text
first of all, thanks for making this repo public. Secondly, could it be that the response that is contained within the login variable used to be in JSON format, and now it is in html format?
Traceback (most recent call last): File "C:\Users\kanin\Projects\ImmoSpider\Scripts\wg-gesucht-crawler-cli-script.py", line 33, in <module> sys.exit(load_entry_point('wg-gesucht-crawler-cli==0.2.1', 'console_scripts', 'wg-gesucht-crawler-cli')()) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 722, in __call__ return self.main(*args, **kwargs) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 697, in main rv = self.invoke(ctx) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 535, in invoke return callback(*args, **kwargs) File "c:\users\kanin\projects\immospider\lib\site-packages\wg_gesucht\cli.py", line 107, in cli wg_gesucht.sign_in() File "c:\users\kanin\projects\immospider\lib\site-packages\wg_gesucht\crawler.py", line 100, in sign_in if login.json() is True: File "c:\users\kanin\projects\immospider\lib\site-packages\requests\models.py", line 897, in json return complexjson.loads(self.text, **kwargs) File "C:\Python39\lib\json\__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Stopped running!
Hi,
first of all, thanks for making this repo public. Secondly, could it be that the response that is contained within the login variable used to be in JSON format, and now it is in html format?
These are the line causing the error:
https://github.com/grantwilliams/wg-gesucht-crawler-cli/blob/56f1263ece4529164d353419581e4115d6616c93/wg_gesucht/crawler.py#L100
This is my call stack:
Traceback (most recent call last): File "C:\Users\kanin\Projects\ImmoSpider\Scripts\wg-gesucht-crawler-cli-script.py", line 33, in <module> sys.exit(load_entry_point('wg-gesucht-crawler-cli==0.2.1', 'console_scripts', 'wg-gesucht-crawler-cli')()) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 722, in __call__ return self.main(*args, **kwargs) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 697, in main rv = self.invoke(ctx) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "c:\users\kanin\projects\immospider\lib\site-packages\click\core.py", line 535, in invoke return callback(*args, **kwargs) File "c:\users\kanin\projects\immospider\lib\site-packages\wg_gesucht\cli.py", line 107, in cli wg_gesucht.sign_in() File "c:\users\kanin\projects\immospider\lib\site-packages\wg_gesucht\crawler.py", line 100, in sign_in if login.json() is True: File "c:\users\kanin\projects\immospider\lib\site-packages\requests\models.py", line 897, in json return complexjson.loads(self.text, **kwargs) File "C:\Python39\lib\json\__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Stopped running!