search

AdguardTeam / AdguardForiOS

The most advanced ad blocker for iOS
https://adguard.com/
GNU General Public License v3.0
1.4k stars 201 forks source link

New algorithm suggestion for updating content blockers #2250

Closed brknstrngz closed 7 months ago

brknstrngz commented 8 months ago

Problem Description

With each of the available Safari content blockers hosting up to 150,000 rules, most user configurations will be fine. However, some people, like myself, would prefer subscribing to lists that exceed that limit. That is currently impossible despite the remaining 5 content blockers having ample space to store the difference.

Proposed Solution

I propose the following algorithm:

  1. Load up all user configured filters into an associative data structure (tree, hash map, either works, really), discarding any duplicate keys (assuming the filter string is the key). Optionally, increment a "discarded filters" counter for each filter subscription, reporting the sum of the counters against the total number of filters from all subscriptions - e.g. "X filters loaded, Y discarded as duplicate".
  2. If the deduplicated filter list exceeds 900,000 rules, return an early error. Otherwise, traverse the data structure in-order, allocating the first 150,000 rules to the first content blocker, the following 150,000 rules (or however many are left) to the second one and so on until the end of the data structure is reached.

Alternatives Considered

With the proposed change, each content blocker would be just a slot for loading filters of all types, as opposed to the current model where each blocker has a designated purpose (either security, adware, annoyances or locale-specific filterign). But perhaps the behaviour (current vs proposed) could be flipped by a switch?

Thoughts?

maxikuzmin commented 7 months ago

@brknstrngz first of all, 150k limit does not work due to the iOS 17 bug.

Second issue is that rules in content blocking filter lists may influence each other and you cannot simply distribute them among different content blockers, this will break the inner logic (the simple example: exception rules will stop working and you'll get a bunch of broken websites).

maxikuzmin commented 7 months ago

@brknstrngz at this time, I'm closing this issue. If you have any more questions, please let me know