search

toby-p / rightmove_webscraper.py

Python class to scrape data from rightmove.co.uk and return listings in a pandas DataFrame object
MIT License
252 stars 112 forks source link

Commercial property #21

Closed ClaudeNovaMB closed 4 years ago

ClaudeNovaMB commented 4 years ago

Hi there, This is a great package ! I have tried property type as commercial which doesnt work. Any pointers?

toby-p commented 4 years ago

Thanks! Could you let me know if you get an error or just no results?

ClaudeNovaMB commented 4 years ago

An error. Invalid rightmove URL

Residential searches work perfectly

toby-p commented 4 years ago

Ok thanks - to be honest I never tried commercial searches before. I think there's an assertion statement that raises that error, so it might just be a case of tweaking that, but will have to test if the xpaths are all the same for commercial results. Will look into it.

ClaudeNovaMB commented 4 years ago

Yeah, line 112 (function rent_or_sale) checks "https://www.rightmove.co.uk/ property-for-sale /..." Commercial url is slightly different "https://www.rightmove.co.uk/ commercial-property-for-sale /...."

ClaudeNovaMB commented 4 years ago

xpath for price is still relevant

ClaudeNovaMB commented 4 years ago

Seem other xpaths look great too

toby-p commented 4 years ago

Cool, thanks. Feel free to submit a pull request if you think you have a fix.

ClaudeNovaMB commented 4 years ago

Let me know if you received the request and if your test result was a success.

toby-p commented 4 years ago

Apologies for the delay - have now implemented this functionality. Hope this helps!