search

w3c / webdriver

Remote control interface that enables introspection and control of user agents.
https://w3c.github.io/webdriver/
Other
679 stars 194 forks source link

URL blocking / ad blocking #1234

Closed alexanderadam closed 5 months ago

alexanderadam commented 6 years ago

The webdriver standard represents a part of the web testing activity. And developers want to test things that are common things that happen in the real life.

One of the most common things in the web is, that people control is the display of ads (or other elements / scripts such as webminers) with the help of adblockers like uBlock Origin and similar ones. Browsers like Firefox, Google Chrome and Chromium, Brave, Safari or Opera even have functionality like this integrated natively (the same goes for many testing solutions, i.e. when using Chrome 84 or newer or puppeteer).

Since this is so common, it should obviously be tested properly. Developers want to know how an application or website reacts when people are visiting it with adblock functionality enabled. That is why I believe it should be integrated in the webdriver standard as well.

andreastt commented 6 years ago

I’m not entirely certain what this issue is regarding?

alexanderadam commented 6 years ago

I guess defining a standardized way of blocking elements in user agents.

I'm not sure how this would look like as well but @barancev suggested opening an issue for webdriver regarding content blocking and I thought you folks might rather know what to do. :wink:

andreastt commented 6 years ago

We can certainly consider it, but I’m not sure there’s a good way of defining what it means to block a URL in web platform terms. It might be possible for resources loaded in HTML, but not generally for any UA resource.

shs96c commented 6 years ago

I'm not sure how meaningful this is. Most of the drivers out there allow you to install extensions (such as adblockers), so there's no need to bake it into this specification. There are some browsers implementing some form of cookie control or adblocking, and the developers of those browsers can use this spec "as is".

hugobast commented 6 years ago

Going to add to this conversation with the why? (for me). I think this blog post from ThoughtBot does a really good job of explaining why we might want this feature.

black-snow commented 6 years ago

As for me the WHY is to not generate $$ for Google when testing pages containing google maps. IMHO this is a must.

/edit: How is this to be considered off-topic? Its a common case that you do not want to hit certain things when testing and it'd be nice if you didn't have to set-up and use a proxy or to manipulate your DNS etc.

JG127 commented 4 years ago

My use case is about blocking a ridiculously huge background image. The test suite has to download it over and over again.

alexanderadam commented 4 years ago

Most of the drivers out there allow you to install extensions (such as adblockers), so there's no need to bake it into this specification.

  1. Well, Selenium is following the the W3C webdriver and doesn't integrate it, because it's not specified.
  2. I would also be open for a specification that describes the integration of extensions. But I have the feeling that this might be much more complex.
JG127 commented 4 years ago

What I could imagine is that the driver would map urls to different resources. This way the test could use a test version of JS, download lighter static content from a local source, similar to mocking by simulation of an external source...

StephanBijzitter commented 3 years ago

This would also make testing of offline functionality in PWAs much more easy. In my specific case, I've wrapped all my fetch calls into a helper function that looks a lot like: return window.__offline ? new Response(...) : fetch(...), but that doesn't stop external scripts, images or the service worker from using the network, so it's a bit... meh.

Dr10s commented 2 years ago

Just use --host-resolver-rules for chrome.

alexanderadam commented 2 years ago

Just use --host-resolver-rules for chrome.

This repository is not about certain ways to achieve this with a particular driver. This repository is about defining WebDriver standard and it describes a platform- and language-neutral wire protocol to remotely instruct the behaviour of web browsers.

alexanderadam commented 6 months ago

It's 2024 now and I stopped using Selenium in all of my projects and switched to other drivers. However, I think that this issue is still relevant — maybe now more than ever — especially since blocking ads is nowadays a widely used feature in pretty much every user agent.

Or how it's put in this article:

According to a survey of 2,000 Americans conducted by research firm Censuswide, on behalf of Ghostery, a maker of software to block ads and online tracking, 52 percent of Americans now use an ad blocker, up from 34 percent according to 2022 Statista data.

And while blocking ads was mainly just supported via extensions when I created this issue in 2018, it is nowadays even integrated natively in certain user agents.

whimboo commented 6 months ago

@alexanderadam In WebDriver BiDi we allow to add network interceptions which can be used to block certain network requests and as such for blocking elements to be loaded at all. Does that satisfy your needs?

Please note that such a feature cannot be done for the WebDriver classic protocol due to limitations of the protocol.

whimboo commented 5 months ago

I'm going to close this issue in favor of the mentioned network interception feature of WebDriver BiDi. If that doesn't fit your needs please explain what's missing. Thanks.