voe.sx: popup

[gettysburg]

Prerequisites

• [X] This is not a support issue or a question. For support, questions, or help, visit /r/uBlockOrigin.
• [X] I performed a cursory search of the issue tracker to avoid opening a duplicate issue.
• [X] I am running the latest version of uBlock Origin (uBO).

I tried to reproduce the issue when...

• [X] uBO is the only extension.
• [X] uBO uses default lists and settings.
• [X] using a new, unmodified browser profile.

Description

The video file hosting site voe.sx added JavaScript code to their video playback page, which tries to open a popup for 3 times before giving up, which is closed by Firefox' or uBlock's anti popup functionality almost instantly.

Due to voe.sx being popular among people uploading copyrighted movies or series, they switch domains often, as sometimes their domains get shut down either by their registrary or by DnS filtering from by ISP's and VPNs, so they change sites often.

I did not search for other mirror, but if you know of one, add it to the list.

URL(s) where the issue occurs.

https://guidon40hyporadius9.com/zpwgcohyfd57
https://voe.sx

Screenshot(s)

No response

uBO version

1.46.0

Browser name and version

Firefox 102.7.0esr (64-bit)

Settings

Enabled all filter lists.

Notes

No response

[gettysburg]

They once again rotated domains, I now get:

https://cyamidpulverulence530.com/b78j20fhpxv1

with the same popups.

I think for this particular site, a filter should be written that recognizes a playback site from voe.sx regardless of domain, maybe by creating a hash of one of their unique website elements - the logo perhaps or by searching for a specific string, otherwise this will stay a reoccurring issue.

[stephenhawk8054]

I don't get popups. The domain is already added

Screenshot:


Can you force update the filter lists and test again?

[gettysburg]

Done, seems fixed - bad timing on my end then :-)

I'll post again once I come across another, not yet listed site.

Thanks.

[gettysburg]

New domain, new popups:

https://boonlessbestselling244.com/2mp7yzdtq1kj

I force-updated all filters twice before visiting the site, but please note that these popups are a bit different.

The original popups always appeared when visiting a voe.sx stream page and clicking anywhere in the page, now they seem to have added code which by randomness determines if you get the original popups (3 in a row), or the new ones with a timer which behave as follows:

You can click anywhere or even start watching, but then, after some timer expires, for me it was 10 or so minutes after watching, I stopped the stream because I had to check on something, and instantly got a popup. I then started the stream again, and didn't get a popup.

One would assume they would give up eventually but they seem to keep rewriting their advertisement code.

[ghajini]

added https://github.com/uBlockOrigin/uAssets/commit/afe9ae8e63eef9d960202c5f4229fd7225e9caef

[gettysburg]

Thank you @ghajini :)

[gettysburg]

New domain has been acquired by VOE, and I am getting the usual popups, please add it to the filter list:

https://matriculant401merited.com/itbrny1kmy24

Thank you!

[mapx-]

fixed 5 hours ago

[gettysburg]

@mapx- You guys are quick, thank you for the swift response and fix.

Off-topic:

If there was only an advanced setting in uBlock Origin to override all filters expiry time globally, so that every X hours all filters would be re-downloaded, I already suggested something like that to @gorhill but he closed the request, saying it would be an unnecessary load on the CDN's which host the filter lists which I can understand, but the load could be drastically lowered by adding a "last-modified" timestamp to all filter lists, and if that timestamp differs from the one in the list that uBlock Origin has cached locally, only then a re-download would be allowed, along with a sensible yet still usable lower cap on the variable which controls the override of the filters "expires in" value.

If that is not enough, uBlock Origin could also cycle and thus distribute the load between the different CDN's, most filter lists have at least 2-3 CDN links as far as I have noticed, along with the setting hidden in the advanced settings.

It really shouldn't be that bad, especially since filter lists are mostly text and you can achieve extremely high compression rates.

This would certainly reduce the cases of issues being opened or comments being added for requests that already have been fixed, I truly believe that this would be the future.

Partial, compressed downloads would also be an option..

Anyway, thanks for your work, and if you have the opportunity, please suggest both ideas to @gorhill.

Or, well, he should get a notification because of me tagging him anyway.

Cheers.

[stephenhawk8054]

ublock is running with 3-4 CDNs already.

[garry-ut99]

If there was only an advanced setting in uBlock Origin to override all filters expiry time globally, so that every X hours all filters would be re-downloaded, (...)

"quicker interval of filter lists update in uBO" requests were already declined several times in the past (including the last week):

• https://github.com/gorhill/uBlock/issues/1333#issuecomment-178706420
• https://github.com/gorhill/uBlock/issues/3117
• https://github.com/uBlockOrigin/uAssets/issues/6526#issuecomment-551524711
• https://github.com/uBlockOrigin/uBlock-issues/issues/2199#issuecomment-1205490004
• https://github.com/uBlockOrigin/uBlock-issues/issues/1165#issuecomment-1441591772

[gettysburg]

I am aware.

[stephenhawk8054]

I am aware.

About the CDNs, ublock is currently running on 4 different hosts/CDNs, and the data from Cloudflare (1 of the 4 hosts) shows over 110TB/month already (multiply by 4 and you get the number). 1 of the 4 is github pages and you can search how much magnitude the current bandwidth exceeds free account's limit.

[mapx-]

We'd need a p2p implementation (not for .. mv3)

[gettysburg]

@mapx- To distribute the lists and take load off the CDN's?

As I said before, since the filter lists are almost only text, you can very quickly compress them using a high compression level.

I highly doubt that any proper CDN has issues distributing a few kilobyte big text files, maybe even just bytes if we are talking about partial updates - even if, theoretically, 1/3 of all uBlock Origin users would activate the advanced settings and override the expiry time.

The load can be further reduced by choosing a sensible minimum override value, such as 1 day.

P2P would mean that anyone could send you a filter list or partial list of changes to apply to said filter list, without any validation, as well as exposing your IP address.

I can also see a P2P implementation without any validation by an authority being abused for zero-day exploits in either uBlock Origin or Firefox, it wouldn't be the first time, except that no P2P implementation was used in the following article:

https://portswigger.net/research/ublock-i-exfiltrate-exploiting-ad-blockers-with-css

[ghajini]

you may use no popups switch for such sites until its fixed in ublock filters, as simple as it can be😊

[garry-ut99]

A: If there was only an advanced setting in uBlock Origin to override all filters expiry time globally, so that every X hours all filters would be re-downloaded

B: This would certainly reduce the cases of issues being opened or comments being added for requests that already have been fixed

It doesn't seem so (B), because there is a contradiction / discrepancy here between A and B, because settings hidden in advanced settings, are used only by a smaller group of advanced users / developers, and majority of issue reports are done by unadvanced users. To benefit majority, a setting must be not hidden in advanced settings, and must be enabled by default, but all this must be without abusing bandwith of servers which is the biggest issue currently.

[gettysburg]

It doesn't seem so (B), because there is a contradiction / discrepancy here between A and B, because settings hidden in advanced settings, are used only by a smaller group of advanced users...

You are absolutely right, of course most users just install the extension and let it do it's work, the default advanced settings are, in my eyes, a bit conservative but do their job.

Maybe if this gets implemented correctly (and by this I mean adding a last-updated field to each list, and only downloading diffs, not entire list files), which also means working together with the CDN providers and the owners of the machines running the web-servers which are ultimately distributing the compressed, partial list updates, I can even see this as a non-hidden setting, visible for everyone.

Sure, clicking "Purge all caches" and then manually re-downloading all lists works too, but if it can be automated, while potentially saving lots of traffic then.. why not?

Partial, compressed updates are the future, in my opinion.

you may use no popups switch for such sites until its fixed in ublock filters, as simple as it can be😊

Thank you for your suggestion but I am not sure how effective that would be, as the popup works by creating an invisible but clickable overlay across the entire document, and it is not that bad, either Firefox or uBlock Origin takes care of it by closing the new window before any content has the chance to load, anyway.

Thank you @ghajini and @garry-ut99 for your input!

[gettysburg]

Getting popups again on:

https://scatch176duplicities.com/urqo2oytr1h5

I re-downloaded all lists before.

[stephenhawk8054]

That site was added to the list days ago already, and I can't reproduce any popups: https://user-images.githubusercontent.com/66517106/227822875-65831547-7117-4610-a910-9815ed189bd0.png

[gettysburg]

I once again purged caches and re-downloaded all filters using uBO 1.48.1b2, and still get popups on the site and on sites embedding it, could it be an issue introduced in the developer build?

Are you using the stable build?

I am running Firefox ESR 102.9.0esr (64-bit) with only uBlock Origin, Tampermonkey and a plugin to suppress / strip the forced DNT Request-Header in private windows.

I'll try to investigate further.

[stephenhawk8054]

I tested with stable build.

uBO 1.48.1b2

There were some code-refactoring in dev builds recently related to script injections, and it can cause some fails with them. The latest version is 1.48.1b5, can you update to the latest dev build version and test again?

https://github.com/gorhill/uBlock/releases

[gettysburg]

I thought Firefox already auto-updated, at least thats how I configured it, to check for extension updates every 6 hours and install them automatically.

I have an appointment right now, you will hear back from me in 2 hours.

[gettysburg]

I noticed some very weird behavior regarding the debug builds.

First, as suggested I backed up my settings and downgraded to 1.48.0. the latest available stable version, and this did fix the popups.

There must be a regression in uBlock Origin somewhere after 1.48.0, which is only confirmed by the change-logs that say script injection code has been refactored, leaving it either broken completely, or only partially in certain scenarios.

I also tried 1.48.1b5, twice - first before downgrading to 1.48.0, which didn't help, the pop-ups were still there, then installed it again after downgrading to stable (1.48.0) and noticing the popups were gone - the second time I installed it, for some reason, there were no popups at all.

I am 100% sure that I did not do any mistakes while removing and installing the developer builds, and my other extensions can't explain or cause this behavior as well.

Also, this is kind of off-topic, but:

@gorhill

Can we please get MVPS hosts file and the list called "AdGuard Tracking Protection filter" added back as standard options in uBlock Origin?

After the removal from uBO (the reason was "list inactivity" I remember), I manually imported them 2 months ago and monitored the lists for a while - both lists are still receiving updates and are getting modified at least every 36 hours, and still block a lot of crap that the other lists don't cover - I think people who don't follow the development of your extension as closely as I do would still benefit a lot from them.

Here are the individual direct URL's from my uBO backup, I'm not sure if they are mirrored by some CDN still, at least I couldn't find a link:

"importedLists": [
      "https://filters.adtidy.org/extension/ublock/filters/3.txt",
      "https://winhelp2002.mvps.org/hosts.txt"
]

Thank you @stephenhawk8054 for confirming the domain is already covered, and everyone involved, you, Stephen, included, for fighting the good fight in general - but I still can't explain the behavior of the latest developer build, however I am convinced that it definitely was no "the problem is sitting behind the screen" situation.

P.S: If you got multiple notifications or e-mails regarding this answer, it was my mistake - sorry about that.

[gettysburg]

After a short appointment and break, I now wanted to continue to watch that movie, and was greeted by the "3 popups in a row" which instantly get closed by either FF or uBO.

I am using the latest developer build, 1.48.1b5, and will probably downgrade to 1.48.0 again until this is resolved.

Please note that I had the site open in the background for quite a while, probably over 1 hour - a hard refresh (CTRL + F5) or fresh loading and the following user interaction with the site does not trigger popups, so there is definitely some regression.

@gorhill

[gorhill]

I can reproduce a difference between 1.48.0 and 1.48.1b5. The curious thing is that the scriptlet which makes a difference has not been touched since it's part of web_accessible_resources and I didn't touch this code. When I except the nowoif filter in 1.48.0, I get the same behavior as in 1.48.1b5. I confirm though that the nowoif scriptlet is really injected, so this needs more investigation.

---

Fixed the issue but now GitHub is broken.

[gettysburg]

@gorhill Thank you Ray for the fast answer and solution, could you please give a statement regarding my off-topic part?

The lists were removed for inactivity, which is simply not true, I sometimes see 100+ cosmetic / network filters added to "AdGuard Tracking Protection filters", and MVPS Hosts file also still gets modified regularly, even worse, only about ~15000 filters from the 40000 total in the "AdGuard Tracking Protection filters" list are already contained in other lists, when it comes to MVPS hosts file, only a single (1) filter out of 9000 is already listed somewhere else.

As mentioned before, after the removal I manually re-imported these two filter lists, and kept an eye on sites using the logger, and can confirm that I still get a lot of hits from both lists, especially the AdGuard one - I think there would be no issue adding them back, if you are unsure, I can contact both maintainers but the fact that a lot of trackers are still filtered by both lists, and only those two lists, without breaking anything, speaks for itself.

What's your opinion?

[gorhill]

MVPS was last updated more than 2 years ago, we don't recommend using unmaintained lists, let alone putting such list as stock one in uBO.

For AdGuard Tracking Protection, these things are discussed internally with experienced filter list maintainers, and this is being discussed. There is no point trying to keep bringing the topic to me. You can import it if you want to use it: https://filters.adtidy.org/extension/ublock/filters/3.txt.

[gettysburg]

I wasn't suggesting adding them as stock and enabling them by default, but adding them as an option for users with an increased need for privacy, but leaving the choice to the user - if you say MVPS wasn't updated in two years then either I have severe optical hallucinations or it is still being maintained, to some extent, despite the

Updated: March-06-2021

line in the block comment at the beginning of the list, since the number of entries grows quite often.

I highly appreciate that you are internally discussing adding AdGuard Tracking Protection back, and just wanted to make clear, as a long time advanced user of uBO, that I vote for the list to be added back, that is all - the final decision obviously has to be made by you and your team.

P.S: I was already using the URL you provided 👍

Thank you for the swift answer and the great extension (which has saved me a lot of headache), anyway.

Now, in an attempt to not further derail this thread, let's leave it to it's original purpose - the ever growing amount of domains of voe.sx.

[gettysburg]

Hello, voe.sx has now added a completely new ad-block detection, and they switched domains once again in order to be able to (briefly) show popups:

35volitantplimsoles5.com

I already purged caches and re-loaded all filter lists but no luck, can't access the video:

Example site: https://scatch176duplicities.com/demhbspjv1de

Note that the example site will just redirect you to the new domain mentioned above, and may stop working entirely soon, on average they burn through 1-3 domains per month.

If the example site doesn't work for you, try https://35volitantplimsoles5.com/demhbspjv1de directly.

Thank you.

P.S:

Since this excessive domain switching has been a re-occurring issue for months now, wouldn't it be smarter, instead of using a domain blacklist for the popups, to switch to a filter that detects if you are on a voe.sx playback page dynamically?

[stephenhawk8054]

You can edit the old comment instead of creating new comment and deleting old one.

[stephenhawk8054]

wouldn't it be smarter, instead of using a domain blacklist for the popups, to switch to a filter that detects if you are on a voe.sx playback page dynamically?

Not possible

[gettysburg]

I am well aware, but due to my OCD, if I have to make edits, I rather delete the posts - but I will now try to keep it to an absolute minimum.

Not possible

Really? I believe you could string-search on a site (as outlined below), or, another idea is checking for the voe_session= cookie that is always set if you are on their playback site.

Maybe you could also use has-text() or has() functions, since the actual playback site did not change in over 2 years now.

[stephenhawk8054]

The point of injecting scriptlets is injecting as soon as possible. By the time uBO knows the response header, it's too late for injecting the scriptlets for defusing the anti-adblock. That's what I understand. Currently you cannot even inject scriptlet based on the parth of the URL. The only way to do dynamically recently are based on the regex of the domain, which is still happens before any connections being fired. If you have any ways to do what you want, please submit PR to help gorhill.

@gorhill Sorry, I wonder if it's possible to injecting scriptlets based on the response header?

---

I am well aware, but due to my OCD, if I have to make edits, I rather delete the posts

Since all the members get notified by any new comments, please just edit it if you want to make changes.

[gettysburg]

This is how it works:

Either a streaming site embeds voe.sx or a link is shared among people, once a user clicks on it, voe.sx then processes the request with the playback ID, then, if the file is found, that request returns status code 302 Found with the (newest, throwaway) playback domain in the Location field:

So before the browser acts on the 302 status code and follows its location, there is plenty of time, if I understood your question properly.

This way you could dynamically detect new playback domains :-)

[stephenhawk8054]

I'm not sure, how many is plenty of time? The only way to know if it works is implementing it. You can try to see if it does.

[gettysburg]

I'm not sure, how many is plenty of time?

150ms according to the developer tools.. not that much, plenty was the wrong word then.

If there is a way to intercept that request though, and thus make the browser not instantly relocate to the new site, we could "borrow" more time though.

I am really not familiar with uBO's code base and it's been years since I've played around with JS / CSS at all, I suggest that a more experienced developer takes a look at this, but I am convinced that there is some way to get such functionality implemented.

I am going to bed now, if there is anything to discuss, expect an answer tomorrow (it's already 4AM / 4:30 in the morning where I live).

Anyway, thanks for the fast commits, I can confirm it gets rid of the popups and ad-block warning. 👍

[stephenhawk8054]

Reading again, I feel bad for pinging gorhill earlier. What you asked for requires uBO to do a search (whatever it is, for status or response header) every time the browser makes a document/iframe connection just to solve this issue (because you want it to do dynamically, not restricted to some domains). This is really inefficient.

[MasterKia]

There is $header already but I'm not sure if it can help here.

[gettysburg]

@stephenhawk8054

A single strcmp, something easy and lightweight, like a === b, takes what, microseconds at best, especially with JS optimizations?

But once again, I'm not an active JS developer.

Take root domain, strcmp to check if we should apply filter to it, done.

uBO must already do this internally, to check whether to apply certain filters matching root domain to websites.

[stephenhawk8054]

You can try to see if it's compatible with filter list compiling, and profile the data to see its performance. We usually underestimate what we think and how it performs when we code.

That being said, looks like you need to develop a new extension dedicated to this site as your desired approach is very different to uBO.

If gorhill is interested in this, he will give his opinions or approach. I won't participate in this any more.

[gorhill]

Keeping track of 302 redirections and injecting scriptlets according to the source of redirects sounds feasible.

The question is how often that situation arise. I probably wouldn't be very motivated to add lots of new code just for one site, but if it happens that this would be useful for numerous distinct cases, I might warm up to implement this.

[gettysburg]

@gorhill For now, only voe.sx comes to mind, but I have seen other streaming sites in the past deploying the same scheme as the aforementioned site:

1.) Playback ID is sent to main streaming platform domain, which does not allow to stream any content for legal reasons. 2.) Backend of main domain then checks if file is available, and returns either a fixed or random throwaway domain (depending on provider) in the Location field along with a 302 Found status code.

If I have time I will take a look at popular streaming sites, and other sites which own many throwaway domains for (mostly) legal and anti-censorship reasons, along with searching through the existing filters, but don't count on it, I am currently ill and not really in the mood.

I am just saying that catching those 302 Found responses is indeed possible, so we can inject scriptlets at the earliest point in time.

[gettysburg]

Hi, they got a new domain again:

https://tummulerviolableness.com