KeyError: 'rate_with_service_fee'

[sebastien247]

Hi, Thanks for this cool tool. I obtain this error on my first launch :

Traceback (most recent call last):
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\utils\defer.py", line 117, in iter_errback
    yield next(it)
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\utils\python.py", line 345, in __next__
    return next(self.data)
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\utils\python.py", line 345, in __next__
    return next(self.data)
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 338, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "c:\users\sebastien\appdata\local\programs\python\python37-32\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "C:\Users\Sebastien\Documents\Studio\airbnb-scraper\deepbnb\spiders\bnb.py", line 90, in parse
    listings = self._get_listings_from_sections(tab['sections'])
  File "C:\Users\Sebastien\Documents\Studio\airbnb-scraper\deepbnb\spiders\bnb.py", line 222, in _get_listings_from_sections
    rate_with_service_fee = pricing['rate_with_service_fee']['amount']
KeyError: 'rate_with_service_fee'

Edit: By comment all lines use "rate_with_service_fee" the script works but not all rooms are scraped.

Thanks Sebastien

[digitalengineering]

Thanks @sebastien247 I've updated the tool to work with the new API, so perhaps this issue is resolved now. it is still a WIP.