search

uBlockOrigin / uBlock-issues

This is the community-maintained issue tracker for uBlock Origin
https://github.com/gorhill/uBlock
945 stars 80 forks source link

Investigate unduly high time-to-readiness as reported in uAssets issue tracker #1865

Closed gorhill closed 11 months ago

gorhill commented 2 years ago

Instances of unduly high time-to-readiness (will add more as more come into view), mobile browsers will be left out for now:

Chromium

Firefox

garry-ut99 commented 1 year ago

Chromium (109 or lower)

maxAssetCacheWait

UPDATE 1:

UPDATE 2 (14-04-2023):

UPDATE 3 (12-06-2023):

UPDATE 4 (05-12-2023):

UPDATE 5 (26-04-2024):

gwarser commented 1 year ago

https://github.com/uBlockOrigin/uAssets/discussions/16939

ghost commented 6 months ago

Investigation continued from https://github.com/uBlockOrigin/uBlock-issues/issues/3217 @stephenhawk8054

default: ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 134679 cosmetic: 46209 scriptlet: 20520 html: 0 listset (total-discarded, last-updated): default: user-filters: 0-0, never ublock-filters: 37572-290, now ublock-badware: 8315-0, now ublock-privacy: 832-0, now ublock-unbreak: 2256-0, now ublock-quick-fixes: 274-3, now easylist: 86177-557, now easyprivacy: 50790-22, now urlhaus-1: 13265-0, now plowe-0: 3736-788, now filterset (user): [empty] userSettings: [none] hiddenSettings: [none] supportStats: allReadyAfter: 3092 ms maxAssetCacheWait: 79 ms cacheBackend: indexedDB ```
with all filters selected from dashboard(excluding regional ones); ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 249501 cosmetic: 250403 scriptlet: 51989 html: 0 listset (total-discarded, last-updated): added: adguard-generic: 80958-6520, never adguard-mobile: 9260-170, never adguard-spyware-url: 1494-120, never adguard-spyware: 88052-28263, never block-lan: 61-0, never curben-phishing: 14001-3, never adguard-cookies: 29275-72, never ublock-cookies-adguard: 1079-1, never fanboy-cookiemonster: 50948-3935, never ublock-cookies-easylist: 1079-1079, never adguard-social: 22829-762, never fanboy-social: 15904-25, never [11 lists not shown]: [too many] default: user-filters: 0-0, never ublock-filters: 37572-290, 7m ublock-badware: 8315-0, 7m ublock-privacy: 832-0, 7m ublock-unbreak: 2256-0, 7m ublock-quick-fixes: 274-3, 7m easylist: 86177-557, 7m easyprivacy: 50790-22, 7m urlhaus-1: 13265-0, 7m plowe-0: 3736-788, 7m filterset (user): [empty] userSettings: suspendUntilListsAreLoaded: true hiddenSettings: [none] supportStats: allReadyAfter: 9873 ms maxAssetCacheWait: 6902 ms cacheBackend: indexedDB ```
with noop rules and hard mode: ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 249501 cosmetic: 250403 scriptlet: 51989 html: 0 listset (total-discarded, last-updated): added: adguard-generic: 80958-6520, 19m adguard-mobile: 9260-14, 19m adguard-spyware-url: 1494-120, 19m adguard-spyware: 88052-28263, 19m block-lan: 61-0, 19m curben-phishing: 14001-2, 19m adguard-cookies: 29275-70, 19m ublock-cookies-adguard: 1079-1, 19m fanboy-cookiemonster: 50948-3935, 19m ublock-cookies-easylist: 1079-1079, 19m adguard-social: 22829-755, 19m fanboy-social: 15904-21, 19m [11 lists not shown]: [too many] default: user-filters: 0-0, never ublock-filters: 37572-630, 18m Δ ublock-badware: 8315-0, 18m Δ ublock-privacy: 832-0, 26m ublock-unbreak: 2256-0, 26m ublock-quick-fixes: 274-3, 26m easylist: 86177-856, 18m Δ easyprivacy: 50790-30, 18m Δ urlhaus-1: 13265-0, 26m plowe-0: 3736-1287, 26m filterset (user): [empty] switchRuleset: added: [array of 14 redacted] removed: no-large-media: behind-the-scene false hostRuleset: added: [array of 70 redacted] userSettings: advancedUserEnabled: true suspendUntilListsAreLoaded: true hiddenSettings: [none] supportStats: allReadyAfter: 7405 ms (selfie) maxAssetCacheWait: 3430 ms cacheBackend: indexedDB ```
after adding custom filters and custom filterlists(including the one mentioned in comments) but without noop rules and hard mode ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 544777 cosmetic: 260389 scriptlet: 52859 html: 0 listset (total-discarded, last-updated): added: https://gitlab.com/magnolia1234/bypass-paywalls-clean-filters/-/raw/main/bpc-paywall-filter.txt: 0-0, never https://o0.pages.dev/Pro/adblock.txt: 155647-45466, 12m https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_trackers_justdomains.txt: 69497-20373, 12m https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate versions Anti-Malware List/AntiMalwareDomains.txt: 21187-295, 12m https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Dandelion Sprout's Anti-Malware List.txt: 85890-1943, 12m https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt: 2621-0, 12m https://raw.githubusercontent.com/gwarser/filter-lists/master/lan-block-strict.txt: 54-0, 12m https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/ultimate.txt: 187303-106751, 12m https://raw.githubusercontent.com/liamengland1/miscfilters/master/antipaywall.txt: 2098-0, 12m https://raw.githubusercontent.com/voidlyvidevoid/filterlist1/main/filterlist.txt: 4-0, 12m https://someonewhocares.org/hosts/ipv6/hosts: 23338-11712, 12m adguard-generic: 80958-10244, 12m [23 lists not shown]: [too many] default: user-filters: 23-0, never ublock-filters: 37577-782, 11m ublock-badware: 8320-49, 11m ublock-privacy: 832-52, 12m ublock-unbreak: 2256-0, 12m ublock-quick-fixes: 274-3, 12m easylist: 86419-33304, 10m Δ easyprivacy: 50793-167, 12m urlhaus-1: 13265-0, 12m plowe-0: 3736-1313, 12m filterset (user): [array of 23 redacted] switchRuleset: added: [array of 1 redacted] userSettings: suspendUntilListsAreLoaded: true hiddenSettings: [none] supportStats: allReadyAfter: 7073 ms maxAssetCacheWait: 3900 ms cacheBackend: indexedDB ```

Surprisingly, I see very low values here. All these tests are done after i make changes and restart the browser. I did check restarting my PC once too and in that case, it showed the higher timings, 32k this time.

stephenhawk8054 commented 6 months ago

@voidlyvidevoid Can you go to chrome://extensions page (type in URL bar) and screenshot it here?

ghost commented 6 months ago

image

stephenhawk8054 commented 6 months ago

@voidlyvidevoid Can you turn off all other extensions and compare the numbers again?

ghost commented 6 months ago

Should I do it again with PC restart each time or is reopening the browser window sufficient? (due to the big difference in the values)

stephenhawk8054 commented 6 months ago

@voidlyvidevoid Just restart the browser is enough

ghost commented 6 months ago

Restarted browser each time using edge://restart

default : ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 134949 cosmetic: 46208 scriptlet: 20523 html: 0 listset (total-discarded, last-updated): default: user-filters: 0-0, never ublock-filters: 37581-137, 1m ublock-badware: 8320-0, 1m ublock-privacy: 832-0, 1m ublock-unbreak: 2256-2, 1m ublock-quick-fixes: 327-1, 1m easylist: 86394-557, 1m easyprivacy: 50790-187, 1m urlhaus-1: 13265-0, 1m plowe-0: 3736-788, 1m filterset (user): [empty] userSettings: [none] hiddenSettings: [none] supportStats: allReadyAfter: 5432 ms maxAssetCacheWait: 4089 ms cacheBackend: indexedDB ```
with hard mode and noop rules: ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 134902 cosmetic: 46211 scriptlet: 20526 html: 0 listset (total-discarded, last-updated): default: user-filters: 0-0, never ublock-filters: 37581-290, 8m ublock-badware: 8320-0, 8m ublock-privacy: 832-0, 8m ublock-unbreak: 2256-2, 8m ublock-quick-fixes: 274-1, 6m Δ easylist: 86394-557, 8m easyprivacy: 50790-810, 6m Δ urlhaus-1: 13265-0, 8m plowe-0: 3736-0, 8m filterset (user): [empty] hostRuleset: added: [array of 70 redacted] userSettings: advancedUserEnabled: true hiddenSettings: [none] supportStats: allReadyAfter: 374 ms (selfie) maxAssetCacheWait: 101 ms cacheBackend: indexedDB ```
with all filterlist from dashboard enabled (excluded the regional) + hard mode with few noop rules ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 249721 cosmetic: 250390 scriptlet: 51995 html: 0 listset (total-discarded, last-updated): added: adguard-generic: 80958-1755, now adguard-mobile: 9260-135, now adguard-spyware-url: 1494-119, now adguard-spyware: 88052-650, now block-lan: 61-0, now curben-phishing: 14001-4, now adguard-cookies: 29275-68, now ublock-cookies-adguard: 1079-1, now fanboy-cookiemonster: 50935-3869, now ublock-cookies-easylist: 1079-1079, now adguard-social: 22829-869, now fanboy-social: 15902-24, now [11 lists not shown]: [too many] default: user-filters: 0-0, never ublock-filters: 37581-3935, 12m ublock-badware: 8320-0, 12m ublock-privacy: 832-0, 12m ublock-unbreak: 2256-5, 12m ublock-quick-fixes: 274-1, 10m Δ easylist: 86394-2326, 12m easyprivacy: 50793-28679, 10m Δ urlhaus-1: 13265-0, 12m plowe-0: 3736-0, 12m filterset (user): [empty] hostRuleset: added: [array of 70 redacted] userSettings: advancedUserEnabled: true suspendUntilListsAreLoaded: true hiddenSettings: [none] supportStats: allReadyAfter: 6864 ms maxAssetCacheWait: 3606 ms cacheBackend: indexedDB ```
above + custom filters added: ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 249722 cosmetic: 250411 scriptlet: 51995 html: 0 listset (total-discarded, last-updated): added: adguard-generic: 80958-6525, 4m adguard-mobile: 9260-14, 4m adguard-spyware-url: 1494-120, 4m adguard-spyware: 88052-28263, 4m block-lan: 61-0, 5m curben-phishing: 14001-2, 4m adguard-cookies: 29275-70, 5m ublock-cookies-adguard: 1079-1, 5m fanboy-cookiemonster: 50935-3935, 5m ublock-cookies-easylist: 1079-1079, 5m adguard-social: 22829-755, 5m fanboy-social: 15902-21, 5m [11 lists not shown]: [too many] default: user-filters: 23-0, never ublock-filters: 37581-631, 16m ublock-badware: 8320-0, 16m ublock-privacy: 832-0, 16m ublock-unbreak: 2256-0, 16m ublock-quick-fixes: 274-3, 14m Δ easylist: 86394-856, 16m easyprivacy: 50793-30, 14m Δ urlhaus-1: 13265-0, 16m plowe-0: 3736-1287, 16m filterset (user): [array of 23 redacted] hostRuleset: added: [array of 70 redacted] userSettings: advancedUserEnabled: true suspendUntilListsAreLoaded: true hiddenSettings: [none] supportStats: allReadyAfter: 3639 ms maxAssetCacheWait: 2057 ms cacheBackend: indexedDB ```
all above + custom filterlists added: (my everyday usage version) ```yml uBlock Origin: 1.57.2 Chromium: 124 filterset (summary): network: 535269 cosmetic: 260047 scriptlet: 52733 html: 0 listset (total-discarded, last-updated): added: https://gitlab.com/magnolia1234/bypass-paywalls-clean-filters/-/raw/main/bpc-paywall-filter.txt: 0-0, never https://o0.pages.dev/Pro/adblock.txt: 155647-61746, never https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_trackers_justdomains.txt: 69497-20354, never https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate versions Anti-Malware List/AntiMalwareDomains.txt: 21187-6166, never https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Dandelion Sprout's Anti-Malware List.txt: 85890-1937, never https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt: 2621-50, never https://raw.githubusercontent.com/gwarser/filter-lists/master/lan-block-strict.txt: 54-53, never https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/ultimate.txt: 187303-113763, never https://raw.githubusercontent.com/liamengland1/miscfilters/master/antipaywall.txt: 2098-14, never https://raw.githubusercontent.com/voidlyvidevoid/filterlist1/main/filterlist.txt: 4-0, never https://someonewhocares.org/hosts/ipv6/hosts: 23338-13776, never adguard-generic: 80958-10241, never [23 lists not shown]: [too many] default: user-filters: 23-0, never ublock-filters: 37443-430, never ublock-badware: 8357-15, never ublock-privacy: 785-0, never ublock-unbreak: 2240-0, never ublock-quick-fixes: 223-0, never easylist: 84420-173, never easyprivacy: 50457-22, never urlhaus-1: 3701-0, never plowe-0: 3758-1263, never filterset (user): [array of 23 redacted] trustedset: added: [array of 6 redacted] switchRuleset: added: [array of 15 redacted] removed: no-large-media: behind-the-scene false hostRuleset: added: [array of 70 redacted] userSettings: advancedUserEnabled: true largeMediaSize: 250 suspendUntilListsAreLoaded: true hiddenSettings: filterAuthorMode: true supportStats: allReadyAfter: 22370 ms maxAssetCacheWait: 13324 ms cacheBackend: indexedDB ```
stephenhawk8054 commented 6 months ago

The allReadyAfter: 5432 ms with default settings is strange, mine is just < 400ms even without selfie using edge://restart

uBlock Origin: 1.57.2
Chromium: 124
filterset (summary):
 network: 134680
 cosmetic: 46211
 scriptlet: 20526
 html: 0
listset (total-discarded, last-updated):
 default:
  user-filters: 0-0, never
  easylist: 86177-559, 1m
  easyprivacy: 50790-813, 1m
  plowe-0: 3759-0, 12d.6h.2m
  ublock-badware: 8315-0, 1m
  ublock-filters: 37581-290, 1m
  ublock-privacy: 832-0, 1m
  ublock-quick-fixes: 274-1, 1m
  ublock-unbreak: 2256-20, 1m
  urlhaus-1: 13265-0, 1m
filterset (user): [empty]
userSettings:
 advancedUserEnabled: true
hiddenSettings:
 filterAuthorMode: true
supportStats:
 allReadyAfter: 391 ms
 maxAssetCacheWait: 106 ms
 cacheBackend: indexedDB
gorhill commented 6 months ago

The allReadyAfter: 5432 ms with default settings is strange

I suspect many of these tests triggered a recompiled of filter lists. To be sure, each test should be done when a selfie is present, that is the normal use case of uBO. Only the second shows usage of a selfie, and the number is as expected, 374 ms.

garry-ut99 commented 6 months ago

Most likely voidlyvidevoid didn't additionally restart browser after reinstalling/resetting uBO to Deafult, before reading the result on Default. He should do so because after reinstalling / resetting uBO to Default / importing backup it always takes longer for uBO to load for the first time (recompilation) - for ex. it takes 1,5s or 2s or even several seconds (on Default without selfie), until browser is restarted - then it's 300-600ms (on Default without selfie). (Of course with selfie it's even faster.)

garry-ut99 commented 6 months ago

UPDATE 5 (26-04-2024): https://github.com/uBlockOrigin/uBlock-issues/issues/1865#issuecomment-1427103640