search

rufusrock / scraping_tools

3 stars 1 forks source link

Fakespot Changing Result Order #9

Closed rufusrock closed 1 year ago

rufusrock commented 1 year ago

fakespot_ab_test_1

It seems like fakespot is manipulating the order of results.

Different Ad Types are rearranged differently.

banner_ad_diff

search_injected_diff

video_ad_diff

carousel_ad_diff

Wondering whether it is possible to reverse engineer the fakespot rearranging algorithm - or would have to rescrape the whole dataset.

rufusrock commented 1 year ago

Sample size was 18 search terms

bramtayl commented 1 year ago

Nice graphs! Well, for one, Fakespot inserts stuff into the window, so it's not too surprising to me that the y-coordinates would change a bit. The carousel ads one is a bit stark though. The "sort by fakespot grade" setting is turned off by default, and assuming you aren't changing that, then I don't think there's anything to do here except to run it once with Fakespot turned on and once with Fakespot turned off. It's probably good to do anyway as a robustness check. And by the way, I strongly suspect we'll have to scrape again a few more times yet with more modifications, because I don't think there was ever a firm plan about what to do with the data anyways. I'm a fan of the "collect a bit of data, play around with it, collect the rest" strategy, but too late for that now, sorry.