Adeeshaj / Carvestor-Scraper

MIT License
0 stars 0 forks source link

Scraper - research sources #2

Closed Adeeshaj closed 9 months ago

Adeeshaj commented 9 months ago

Research the target websites to understand their structure, layout, and how data is presented. Analyze the data you want to collect and determine its location on the websites. Investigate any potential obstacles or challenges, such as anti-scraping measures.

Adeeshaj commented 9 months ago

Target Websites - ikman

Reputable Sources - by google ranking

ikman - https://ikman.lk/ riyasewana - https://riyasewana.com/ patpat - https://www.patpat.lk/ autolanka - https://www.autolanka.com/ autodirect - https://autodirect.lk/

Data Availability

Ikman - Brand, Model, Trim / Edition, Year of Manufacture, Condition, Transmission, Body type, Fuel type, Engine capacity, Mileage, location, date, title, price, Description

riyasewana - Contact, Price, Make, Model, YOM(year), Mileage, Gear, Fuel Type, Options, Engine (cc), Details, title, location, date

patpat - title, location, date, Model Year, Condition, Transmission, Manufacturer, Model, Fuel Type, Engine Capacity, Mileage, Color

autolanka - no recent listings

autodirect- not much informations

here we reject autolanka and autodirect

Website Policies

Website TOS robot.txt
Ikman okay okay
riyasewana no okay
patpat no okay

here we reject riyasewana and patpat only remain is ikman