search

adodd202 / Airbnb_Scraping

Scraping Airbnb with Scrapy Splash and performing EDA in Python and R.
24 stars 12 forks source link

Meta in SplashRequest #2

Closed roberthng closed 3 years ago

roberthng commented 3 years ago

Hey, I'm quite new to scrapy splash but i found your code to be suitable for my future use.

I have a question about the function of meta in in the SplashRequest function in your airbnb_spider.py file

yield SplashRequest(url, callback=self.parse_PriceRange, args = {"wait": 5}, endpoint = "render.html", meta = {"price":price})

Could you explain the function of the meta ={"price":price}? Thank you

adodd202 commented 3 years ago

Ah yeah sure. The price here is the specific price range given to search over - makes more sense when you try a few different URLS similar to line 30 of the same file. As for why it's in the meta, I don't honestly remember - it's been 3 years since I wrote this. You can try it with the meta info and see if it still works as expected. Ultimately, it is intended to specify a price range though so that the scraper can gather more airbnb locations.