search

strawp / autotrader-miner

Single-user site for mining Autotrader.co.uk data and providing more useful searches for cars
19 stars 1 forks source link

Is there any chance on update? #3

Open infowebstudiobg opened 7 years ago

infowebstudiobg commented 7 years ago

It seems autotrader has changed its layout, and Getting colours from autotrader... failed..

shupac800 commented 7 years ago

Autotrader has changed its markup multiple times since I wrote this script. Now, in list view, the results are displayed in a "clickable wrapper" that, because it's an image, can't be easily scraped. I have no plans to update this project.

strawp commented 7 years ago

No idea who shupac800 is. He has nothing to do with this project though.

Colours might now not be an option but most of everything else should be OK with my local copy that I will test and push in the near future.

infowebstudiobg commented 7 years ago

Strawp, is it possible me donating this project and get it working as fast as possible? victor@donchev.com

strawp commented 7 years ago

I've just pushed what I have. It was only a few weeks ago this was happily pulling down car data so it's probably not far off working

infowebstudiobg commented 7 years ago

it seems autotrader is blocking me somehow, i am trying to echo the $txt and it gives me nothing, i tried with adding browser identification, cookies, but no effect. Strawp, whats the chance that you fix the script in the nearest week, cause i will have to be paying to other php dev to fix it..? even after update today it says it can't fetch colors, i assume it doesnt fetch any content at all with file get contents function.

strawp commented 7 years ago

AT will block IPs if they see excessive requests. I wouldn't be attempting to pull down huge amounts of data with this. I tend to have a couple of searches live which might be less than 10 pages each. The script will run several times a day but it won't be drilling into each item of those search results.

This is my top priority project at the moment as I'm looking at cars again, it's simply a lack of time available.

Colour wasn't something I fixed in the latest updates. Let me see if I can push something now though...