Correctness of scraping results

godfriedmeesters / scraper

As part of DiffScraper, one or more bots can be deployed. Ready-to-use bots are provided that can extract offers from mobile applications, mobile websites and desktop websites.

GNU General Public License v3.0

2 stars 0 forks source link

Correctness of scraping results #17

Open bkrumnow opened 3 years ago

bkrumnow commented 3 years ago

[ ] Check initial dataset
[ ] Add automated checks (SQL queries)

bkrumnow commented 3 years ago

@godfriedmeesters These are the last two things needed to run the scan, correct? The first point is something will have addressed within the next days. So, get your scrapers ready, if there are any issues left.

For the automated analysis, I hope that I have some queries we can run after the first check.

godfriedmeesters commented 3 years ago

Detected tsome App scrapers that returned duplicate offers. Should normally be fixed now. (May 16)

EuroWings and AirFrance data is OK for both app and web, already made analysis of these