Generate a benchmark graph for the FAQ page

[mushon]

Use only 3 bars:

• no-blocker
• Adnauseam
• AdBlock Plus

[mushon]

Also, if you can add more fineprint that will validate the methodology of the benchmark that would be great.

[cqx931]

Which of the following shall be included (they are separate graphs thus far)?

Metrics:

1. Number of 3rd-parties contacted
2. Overall memory footprint (MB)
3. Total page load time (sec)

[mushon]

We only need the total page load

[cqx931]

Something like this?

[mushon]

That looks great! Which is actually a problem…

I was thinking of something very dry and direct, coming directly from a spreadsheet, and definitely not-designed (which some may interpret as “doctored") exactly like this one only with the first 3 bars instead of 6:

The order should be:

1. No Bloker
2. Adblock Plus
3. Adnauseam

Thanks.

[cqx931]

[mushon]

Right. Though we had better stats before, no? This doesn’t support the claim that we’re more than twice as fast as with no blocker. Any reason not to use the previous stats? --  Mushon Zer-Aviv Mushon.com | Shual.com | @mushon

On October 28, 2016 at 9:40:47 AM, cqx931 (notifications@github.com) wrote:

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[cqx931]

Yes, the older stats are different. The above graph is from the test I did today and I did have all the right settings in my browser. There could be performance difference though, since in the current workflow we only run each site for 3 times... without blocker, si.com alone can have a performance difference from 10 seconds to 40 seconds to load( most content are loaded, but you can still see the spinning wheel... ) Or we could replace si.com with another test case...

[mushon]

is there some formal set of sites we test on?

[cqx931]

This is the current test list:

http://www.si.com https://search.yahoo.com/search?p=watch http://news.google.com/ http://www.huffingtonpost.com/ http://www.cnn.com/ http://www.nytimes.com/ http://www.foxnews.com/ http://www.nbcnews.com/ http://www.dailymail.co.uk/ http://www.washingtonpost.com/ http://www.theguardian.com/ https://news.ycombinator.com/

[mushon]

Is it based on the 10 most popular sites or something?

--  Mushon Zer-Aviv Mushon.com | Shual.com | @mushon

On October 28, 2016 at 10:54:26 AM, cqx931 (notifications@github.com) wrote:

This is the current test list:

http://www.si.com https://search.yahoo.com/search?p=watch http://news.google.com/ http://www.huffingtonpost.com/ http://www.cnn.com/ http://www.nytimes.com/ http://www.foxnews.com/ http://www.nbcnews.com/ http://www.dailymail.co.uk/ http://www.washingtonpost.com/ http://www.theguardian.com/ https://news.ycombinator.com/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cqx931]

no I don't think so...they are mainly news sites with many visual ads.

[mushon]

I know this is a tricky thing and I know (and kinda like) that this discussion is public, but I wish we could come up with a clear and decent methodology that could still support our (previously supported) claim that using ADN loads pages more than twice faster.

:)

--  Mushon Zer-Aviv Mushon.com | Shual.com | @mushon

On October 28, 2016 at 12:15:49 PM, cqx931 (notifications@github.com) wrote:

no I don't think so...they are mainly news sites with many visual ads.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cqx931]

A list based on most popular sites sounds reasonable for me. But not all of them have ads, or visual ads. And it is also not reasonable to test this on the homepage of a video sites... So maybe we can create a new website list based on either alexa topsites global or the country list for US and then pickup those with visual ads? @dhowe http://www.alexa.com/topsites/global;0

Another thing that I want to mention about this testing workflow is gmail account... Because many ads are provided by google, you tend to see way more ads from all websites if you once logged in with gmail and the browser remembered that...( that can also be a possible reason why Cyrus has a longer loading time by no blocker) Not sure whether we should include or exclude this in our settings.

[dhowe]

can we experiment with some different values for these parameters?

[cqx931]

Alexa Global top 50 sites

I ticked some sites that I find reasonable to test with, but many of them need to be logged in(mainly for social media)

• [ ] 1.Google.com (Text)
• [x] 2.Youtube.com (Video & Image)
• [ ] 3.Facebook.com(Post & Image)
• [x] 4.Baidu.com(Text & Image)
• [ ] 5.Wikipedia.org(No Ad)
• [x] 6.Yahoo.com(Image)
• [ ] 7.Google.co.in
• [x] 8.Twitter.com(Post & Image)
• [x] 9.Amazon.com(Image)
• [x] 10.Qq.com(Image)
• [ ] 11.Google.co.jp
• [ ] 12.Live.com(No Ad)
• [ ] 13.Taobao.com(trackers)
• [x] 14.Linkedin.com(Image)
• [ ] 15.Vk.com(Russian)
• [ ] 16.Instagram.com(No Ad)
• [x] 17.Hao123.com(Images & links)
• [x] 18.Sohu.com(Image)
• [x] 19.Sina.com.cn(Image)
• [ ] 20.360.cn(no obvious ad on homepage)
• [x] 21.Weibo.com(Image)
• [ ] 22.Google.de
• [x] 23.Tmall.com(Image)
• [x] 24.Reddit.com(Image)
• [ ] 25.Google.co.uk
• [ ] 26.Google.com.br
• [ ] 27.Yahoo.co.jp
• [ ] 28.Google.fr
• [ ] 29.Yandex.ru(Russian)
• [ ] 30.Google.ru
• [ ] 31.Onclickads.net(Ad company...)
• [ ] 32.T.co
• [ ] 33.Ebay.com(Tracker)
• [ ] 34.Soso.com(Search Section)
• [ ] 35.Wordpress.com
• [ ] 36.Bing.com (Text)
• [ ] 37.Blogspot.com(Tracker)
• [x] 38.Tumblr.com(Image)
• [ ] 39.Google.it
• [x] 40.Stackoverflow.com(Image)
• [ ] 41.Google.es
• [x] 42.Msn.com (Image)
• [x] 43.Gmw.cn(Image)
• [ ] 44.Microsoft.com(No obvious ad)
• [ ] 45.Aliexpress.com
• [ ] 46.Apple.com (No Ad)
• [ ] 47.Pinterest.com (No Ad)
• [ ] 48.Google.com.mx
• [ ] 49.Netflix.com
• [ ] 50.Paypal.com

[dhowe]

status, @cqx931 ?

[cqx931]

I thought we are going trying to agree on "a clear and decent methodology that could still support our (previously supported) claim that using ADN loads pages more than twice faster."...

My opinion is to get rid of http://www.si.com at least, and get a longer test list with more repetitions. And I can see whether my test result still support the claim or not.

[dhowe]

sounds good

[cqx931]

So here comes a new test result with the top 15 news websites that was also used as an example in sessbench.

When it comes to the number, this still can't support the twice faster statement and it is actually slower than the last test I did.The fix of header for the cookies ticket could potentially influenced the performance.

[cqx931]

As for the testing procedure I used sessbench and borrowed some code from the "Page Load Time" extension to get the load time result. Nightwatch could also be helpful for the automatization of this metric workflow. I can check this after finishing the collecting script.

[dhowe]

What browser settings are used in the above test? Flash, 3rd-party cookies, etc?

[cqx931]

flash and 3rd party cookies are both disabled according to the metric ticket. Before each test, the browsing history is also cleaned up.

[dhowe]

I'm curious how enabling 3rd-party cookies (the browser's default) affects the results?

[mushon]

I also think it makes sense to turn on Flash, as some sites may use Flash ads and a huge majority of browsers are still Flash enabled.

Mushon Zer-Aviv Mushon.com | Shual.com | @Mushon

On Nov 16, 2016, at 20:03, dhowe notifications@github.com wrote:

I'm curious how enabling 3rd-party cookies (the browser's default) affects the results?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[dhowe]

all similar testing by addons/extensions is done with flash disabled (adds too much uncertainty as to what results mean)

[cqx931]

3rd-party cookies do have a slight effect on the result, but it is quite tiny on a testing list of 15 websites. In average, I can see 3 seconds more loading time for no blocker and roughly 1 second more for AdNauseam. Flash doesn't really have an influence on this specific testing list, as these sites don't show ads in flash format.

[dhowe]

For the full-set of benchmark comparisons we might (some day) also include Opera's built-in blocker...

[dhowe]

Lets try to get these posted in the FAQ soon

[mushon]

I changed the wording in the press release to: "and nearly twice faster than with no ad blocker installed"

[mushon]

let's keep benchmarks to the FAQ page

[cqx931]

a latest benchmark graph

Date:2017.01.06 AdNauseam Version: 3.0.2 Ad Block Plus Version:1.12.4

Settings: Both Flash and third-party cookies are disabled List of Website: http://news.yahoo.com/ http://www.cnn.com/ http://news.google.com/ http://www.nytimes.com/ http://www.foxnews.com/ http://www.theguardian.com/ http://www.nbcnews.com/ http://www.dailymail.co.uk/ http://www.usatoday.com/ http://www.washingtonpost.com/ http://www.wsj.com/ http://www.abcnews.go.com/ http://news.bbc.co.uk/ http://www.latimes.com/

I did this one in school, so in general the total loading time seems to be longer than the one I did at home. But the percentage is very similar to the last one.

[dhowe]

don't like how close we are to abp (seems worse than i remember) ... are any sites particularly bad or good for adn?

[cqx931]

The following sites always take a relative longer time to load, (almost the same or even a bit longer than no blocker) http://www.abcnews.go.com/ http://www.nbcnews.com/ http://www.cnn.com/ http://www.foxnews.com/

What are the exact things that make adnauseam load faster? It's quite straight forward for me the uBlock's page load time is shorter, as it blocks many requests... We do block beacon and some other requests, but at least for these pages, requests are rarely blocked. Will hiding ads save some rendering time?

[dhowe]

possibly, but the big time wins comes from blocks...

Strategy: identify requests that are blocked by ublock but not adn, that don't result in visual ads, then add rules for these to adnauseam.txt (keeping high-priority here)

[cqx931]

https://github.com/dhowe/AdNauseam/issues/740

[dhowe]

how does this affect the metrics?

[cqx931]

Quite a lot. This time it looks much better.

[dhowe]

"Ad Block Plus" should be "Adblock Plus". Can you update and then add the image to the FAQ (with a note about what we are testing) to this question @cqx931

[cqx931]

Added, please check https://github.com/dhowe/AdNauseam/wiki/FAQ#what-is-adnauseams-performance-like-will-it-speed-up-or-slow-down-my-browsing

[cqx931]

One more try with AdBlock

Date:2017.01.13 AdNauseam Version: 3.1.20 Adblock Plus Version:1.12.4 AdBlock Version: 3.8.4 Result after improving some blocking rules here https://github.com/dhowe/AdNauseam/issues/740 I also turned off ad clicking this time.It can reduce a few seconds from the total page loading time... Almost 50% for now

[cqx931]

close for now, reopen when needed.