Duplicate filters check thread

[ghajini]

Remove some duplicates found in ublock filters and ublock unbreak list as compared with easylist ....

||adop.pw^$3p
||peethobo.com^$3p
||puntoenun.site^
||d10lv7w3g0jvk9.cloudfront.net^
||d2glav2919q4cw.cloudfront.net^
||swesomepop.com^
||cadsecz.com^
||bursagarc.com^
||dmvbdfblevxvx.com^
@@||google.com/recaptcha/$script,csp

[mapx-]

thx

[ghajini]

Some more duplicates

||sayfibhat.club^$3p
||ig0nr8hhhb.com^$3p
||riverhit.com^$3p
||adcalm.com^$3p

[ghajini]

||wpu.sh^$3p
||geepwnfly.club^
||dayadopen.site^
||6ped2nd3yp.com^
||surfe.pro^$3p
||mediaoaktree.com^$3p
||cdn-server.top^$3p
||adser.io^$3p

[mapx-]

https://github.com/uBlockOrigin/uAssets/commit/e4aea72197da22f6f78dbf056c0dd1b77cb34bd4

[kulfoon]

57 duplicates in uBlock filters list as compared with easylist:

``` @@||dailymail.co.uk^$ghide @@||msn.com^$ghide ||2b815136jjjv.com^ ||4prgkoj870j9.com^ ||8ue4.xyz^ ||adworkmedia.net^$3p ||aenan.pw^$3p ||aigneloa.com^$3p ||ailawlani.club^$3p ||aleapeact.club^$3p ||areapebeg.club^ ||artamkari.com^$3p ||artapeare.site^ ||badslopes.com^$3p ||bittraffic.io^$3p ||bl0uxepb4o.com^ ||bordeded.pro^ ||boyaidare.club^ ||busfarmum.com^ ||cadspmz.com^$3p ||cheksoam.com^$3p ||crocopop.com^ ||d18g6t7whf8ejf.cloudfront.net^ ||d18mealirgdbbz.cloudfront.net^ ||dadsokz.com^ ||dumbpop.com^ ||eardewova.com^ ||fatbuydoc.com^ ||fgddrrr.com^$3p ||foazasis.com^$3p ||ftigken.online^$3p ||gadsips.com^ ||gld35wndkn.com^$3p ||gstralpolita.info^ ||jphlsnzlxdz.com^$3p ||meloads.com^$3p ||menuladshy.life^ ||ml0z14azlflr.com^ ||mostviewed.xyz^$3p ||myopenads.com^$3p ||numibit.net^$3p ||oevqxivhquovq.com^ ||performanceonclick.com^ ||pldsafpulgnw.com^$3p ||popnc.com^$3p ||postlnk.com^ ||processaryen.site^ ||proglack.site^$3p ||qmkr7isp3o.com^$3p ||surgermystem.site^ ||thoudroa.net^$3p ||tj9pcnb2ws.com^ ||uzk4umokyri3.com^ ||vdains.info^$3p ||worldmapd.online^$3p ||wpush.org^$3p ||xr8wv50k1daj.com^ ```

[mapx-]

thanks

[ghajini]

||d3c3uihon9kmp.cloudfront.net^
##a[href^="http://azmobilestore.co/"]
||waxpigbaa.com^
||wizliebeg.com^
||d261u4g5nqprix.cloudfront.net^

[mapx-]

thx

[kulfoon]

uBlock filters:

33 Duplicates with EasyList:

``` history.com#@#.m-in-content-ad-row ||1aogoh1j81g3.com^ ||aadajnnn133.xyz^ ||areantaid.site^ ||aukalerim.com^ ||bmkz57b79pxk.com^ ||cadspms.com^ ||cjf25jklrwqt.com^ ||dlfvgndsdfsn.com^ ||dryzapsol.com^ ||extrementtgfa.site^ ||feeohmchi.com^ ||gadsoks.com^ ||kenfeeask.com^ ||repruggob.com^ ||responservbzh.icu^ ||yepsobtax.com^ ||yonatefin.site^ ||adxoo.com^$3p ||hiprofitnetworks.com^$3p ||lofv.xyz^$3p @@||ajax.googleapis.com^$script,domain=rule34hentai.net streamty.com##.video_adp3 youtube.com##ytd-promoted-sparkles-web-renderer nme.com##.ad-container nme.com##.header-advert-wrapper msn.com##.nativeadasideplaceholder msn.com##iframe[id^="_mN_dy_"] msn.com##.nativeadserversidecontentmodule yourlust.com##.banner_right hd-easyporn.com##.vjs-overlay @@||dailymail.co.uk^$ghide @@||msn.com^$ghide ```

36 Rules ^$ redundant by EasyList's rules ^:

``` ||d141wsrw9m4as6.cloudfront.net^$3p ||trblocked.com^$3p ||elfirkelk.com^$3p ||aj2073.online^$3p ||bestvideo2019.xyz^$3p ||gadsecz.com^$3p ||katerigordas.pro^$3p ||rydresa.info^$3p ||artantash.club^$3p ||beiven.pw^$3p ||ladsdown.com^$3p ||ds88pc0kw6cvc.cloudfront.net^$3p ||madsans.com^$3p ||vaipsona.com^$3p ||gadsokz.com^$3p ||dash.tmearn.com^$frame ||padsbrown.com^$3p ||dadskis.com^$3p ||dddomainccc.com^$3p ||fandmo.com^$3p ||ladsabz.com^$3p ||robjamguy.club^$3p ||padsecs.com^$3p ||d2k7b1tjy36ro0.cloudfront.net^$3p ||fixchidie.com^$3p ||onclickgenius.com^$3p ||coinadster.com^$3p ||playerseo.club^$3p ||hadsatz.com^$3p ||meowpushnot.com^$3p ||d2ykons4g8jre6.cloudfront.net^$3p ||yonashbin.com^$3p ||7x7.fun^$3p ||dimfarlow.com^$3p ||arejoysew.com^$3p ||csnd.xyz^$3p ```

Duplicates with EasyPrivacy: ||coinzillatag.com^$3p ||hostcontent.live^$3p ||heymatic.com^$3p

uBlock - Badware risks:

Duplicates with EasyList: ||forwrdnow.com^$document windowsloop.com##.wl-prakatana

uBlock - Privacy:

Duplicates with EasyList: /g00/*/clientprofiler/adb

Duplicates with EasyPrivacy: ||callreports.com^$3p ||salemove.com^$3p

uBlock - Resource abuse:

Duplicates with EasyList: ||d3ahinqqx1dy5v.cloudfront.net^

Duplicates with EasyPrivacy: /coinimp-cache/*

uBlock - Unbreak:

7 Duplicates with EasyList:

``` cadlinecommunity.co.uk#@#.sidebar_advert ||adswizz.com^$3p ||bridgetrack.com^$3p ||valuecommerce.com^$3p @@||cdn.jsdelivr.net^$script,domain=stream2watch.ws @@||thepiratebay.org/static/main.js$script,1p @@||martinfowler.com/articles/asyncJS.css$css,1p @@||nitropay.com/ads-$script,domain=lolalytics.com ```

28 Duplicates with EasyPrivacy:

``` ||2o7.net^$3p ||admitad.com^$3p ||adswizz.com^$3p ||advertising.com^$3p ||amplitude.com^$3p ||awin1.com^$3p ||crazyegg.com^$3p ||evergage.com^$3p ||hotjar.com^$3p ||kissmetrics.com^$3p ||marketo.com^$3p ||mixpanel.com^$3p ||mouseflow.com^$3p ||optimizely.com^$3p ||pagefair.com^$3p ||smartlook.com^$3p ||statcounter.com^$3p ||trafic.ro^$3p ||treasuredata.com^$3p ||woopra.com^$3p ||connatix.com^$3p @@||analytics.edgekey.net/ma_library/html5/html5_malibrary.js$script,domain=mxplayer.in @@||d1z2jf7jlzjs58.cloudfront.net/p.js$script,domain=nfl.com @@||play.history.com/vendor/AppMeasurement.js$script,1p @@||play.history.com/vendor/FacebookPixel.min.js$script,1p @@||play.mylifetime.com/vendor/AppMeasurement.js$script,1p @@||play.mylifetime.com/vendor/FacebookPixel.min.js$script,1p @@||static.lci.fr/assets/scripts/common/user-agent-analyzer.js$script,1p ```

uBlock filters – Annoyances:

No duplicates or redundancies found.

I consider uBO lists reasonable clean compared to what I found in AdGuard Base Filter.

[mapx-]

thanks, great job

[ghajini]

Small Duplicates

||ughhimtoy.com^ ||cogleapad.com^

[ghajini]

some more

||ushoofop.com^ ||betteradsystem.com^$3p ||pophahwot.com^ ||refnippod.com^ ||redresseducatedage.com^ ||qtbb6.com^ ||barfsmiaowpit.com^ ||dealsnbuy.com^ ||truesweb.com^

[ghajini]

||greatdexchange.com^$popup in ublock- unbreak included in Easylist

[mapx-]

I see no greatdexchange in unbreak or other list

[ghajini]

Sorry it is in ublock filters -badware risks

[mapx-]

well, it's ||greatdexchange.com^$all (blocking popup, document, inline-font, inline-script)

[ghajini]

It shows up like this

https://user-images.githubusercontent.com/20338483/93749264-43ca4c00-fc17-11ea-8757-a585d69b438f.png

Fine it's like ...^$all in ublock badware risks😃

[uBlock-user]

It shows up like this

https://user-images.githubusercontent.com/20338483/93749264-43ca4c00-fc17-11ea-8757-a585d69b438f.png

By design. $all is combination of those 4 filter types.

[ghajini]

duplicates as per https://github.com/uBlockOrigin/uAssets/commit/8d2001bfa19f7f3b71757d8d0709560a29f55e79#commitcomment-42265740 i followed

||adpay.network^ ||anthropophile.site^ ||azoaltou.com^ ||bestcontenthost.com^ ||bgyawaib.com^ ||cadsecs.com^ ||cadsimz.com^ ||cpm-ad.com^ ||d13k7prax1yi04.cloudfront.net^ ||d3mr7y154d2qg5.cloudfront.net^ ||dadsabz.com^ ||disloyalmeddling.com^ ||displaycontentnetwork.com^ ||eawp2ra7.top^
||eglates.pro^
||encasedandelionhauled.com^ ||fhgh9sd.com^ ||flushafterwardinteger.com^ ||fosterstrawberry.com^ ||fourthchoseark.com^ ||gagpetwet.com^ ||graxarosi.pro^ ||greedysafely.com^ ||hadsokz.com^ ||hangorientalauto.com^ ||headedtamhemp.com^ ||himizoli.pro^ ||hobfadbig.com^ ||hpyjmp.com^ ||i4tvy8vk9f74.com^ ||illustrationmanlypageant.com^ ||irtyf.com^
||jiltlargosirk.com^ ||judgementinvincible.com^ ||kursatarak.com^ ||ladsans.com^ ||layouttransparent.com^ ||lingeriebarren.com^ ||madslimz.com^ ||manbragshep.com^ ||megmeallab.com^ ||n1ora14.online^ ||naghocmu.com^ ||naustoch.net^ ||padsreds.com^ ||painsko.com^ ||pantherfaucetchoked.com^ ||patithatspartal.pro^ ||poemsdeucewee.com^
||qjjto.com^ ||retamponniez.site^ ||sillrochester.com^ ||socialbars-web4.com^ ||sowhypepse.pro^ ||terra8nb.com^ ||tradblock.com^ ||uziilrlvkiy.com^ ||vendnibtemp.com^ ||wt2noay3fgjn.com^ ||wybutary.xyz^ ||x40u1vj75ks9.com^ ||yowlcapkegs.com^ ||zvmldjxh.com^

[mapx-]

||imgur.com^$image is bad (L.E. fixed)

[ghajini]

yes i was editing ,updated now comments, cross check

[ghajini]

||36b643a6606e.com^     
||ablybeastssarcastic.com^      
||achedgodroad.com^     
||actressact.com^           
||agogdebitape.com^     
||agrarianeruption.com^     
||aidbaronice.casa^     
||andorranisas.xyz^     
||badsecs.com^      
||bcfabscdt.com^            
||bruvwmrxudd.com^      
||cavewaxmalt.com^      
||d36zfztxfflmqo.cloudfront.net^        
||dairouzy.net^     
||denetsuk.com^     
||dlfclcey.com^     
||doufoacu.net^     
||dudslubesviol.com^        
||fdrgtt9edmej010.com^      
||fjioh.com^        
||getpdaiddaily.com^        
||gqtffirr.com^     
||halveclause.com^      
||hoofstepfather.com^       
||houdodoo.net^     
||hqtlvmcrpb.com^       
||hrsstrokecocoon.com^      
||identifyluckyexactly.com^     
||implicationdownloadpercy.com^     
||indecentfaces.com^        
||jomtingi.net^     
||jutyledu.pro^     
||l8ea1nesmb0e.com^     
||leftenvoy.com^        
||lutcdiimmeze.com^     
||madsoks.com^      
||mangelobsolden.cam^       
||mgsqbazssiq.com^      
||minorexportshopping.com^      
||mothhatekick.com^     
||mpnjaifkchgw.com^     
||nellads.com^      
||noonraptnite.com^     
||pawheatyous.com^      
||phooreew.net^     
||podsolnu9hi10.com^        
||pointexcusestretching.com^        
||pregnancydiscreditbee.com^        
||ptairtug.net^     
||pulpsbarndomed.com^       
||pushebrod.com^        
||realmpallduns.com^        
||rompuwit.net^     
||sankaudacityrefine.com^       
||shojovuptu.pro^       
||showmebars.com^       
||tathitch.net^     
||timeruniversalreligious.com^      
||truernewlywait.com^       
||ttgmjfgldgv9ed10.com^     
||web-bars.com^     
||whimsoplynx.com^      
||xiqgqftva.com^        
||yim3eyv5.top^     
||zugnogne.com^

[ghajini]

##.mtvn-mpu-ad      
anyporn.com##.inplb     
informer.com###ad0      
informer.com##.spnsd        
macrumors.com##.tertiary        
||36b643a6606e.com^     
||aj1907.online^        
||alstrome9riya10.com^      
||astra9dlya10.com^     
||dorbanise.pw^     
||dostophog.com^        
||ejirytuw.xyz^     
||glothouw.link^        
||hulkflugarb.com^      
||ifsmatbeg.site^       
||irtyvhrthhya.com^     
||ivfihumeajjmbc.com^       
||ledmophemp.com^       
||mowyappedbibs.com^        
||negyuk.com^       
||plufdsb.com^      
||populationparts.com^      
||rowframemailer.com^       
||sqhneencmysrk.com^        
||umekian.pw^

[ghajini]

ah-me.com##.flirt-block     
hentaistream.com##.othercontent     
sunporno.com##.a-block      
||35.232.188.118^       
||fabrouter.com^        
||gjw5qbyn93b8.com^     
||goawayn0w.com^        
||ladsips.com^      
||moncoerbb.com^        
||neechube.net^     
||seephoak.net^     
||softspace.mobi^       
||sskmnews.pro^     
||trtjigpsscmv9epe10.com^       
||unacceptableclevercapable.com^        
||zeejuthu.com^

[mapx-]

I don't find ||35.3.188.118^

[ghajini]

its ||35.232.188.118^ ,my mishap

[ghajini]

##.amp-ad
||aulteeby.net^
||chivoufe.com^
||freychang.fun^
||jayhintcull.com^
||lgpumcja.com^
||mobcrummymow.com^
||modificationdispatch.com^
||tfzznclztl.com^

[ghajini]

fpo.xxx##.top   
gadgetsnow.com##[data-type="mtf"]   
||aaaaaco.com^  
||aimseerg.com^ 
||grainstucked.com^ 
||guardcathedral.com^   
||jcwuzktevijp.com^ 
||jggqgavrqkudu.com^    
||lilacdefencelessroyal.com^    
||mesucces.top^ 
||padslimz.com^ 
||pedestrianneighbourly.com^    
||pigtisetdilqsi.com^   
||pqtiqpfyeeayjk.com^   
||producingsavourestablish.com^ 
||ptonauls.net^ 
||pyeditingpro.biz^ 
||restights.pro^    
||setbackrelishdivine.com^  
||suptaibo.com^
||touphipt.com^
||unrulyrotate.com^
||whoostoo.net^ 
||writtenorbithalfbaked.com^

[ghajini]

qcock.com##.spot-thumbs
||bitads.io^
||caphaiks.com^
||cdncqw.com^
||cdndqw.com^
||dayanygold.com^
||disgustingvoluntarilykiss.com^
||dmxftmoqz.com^
||dqjazjbqrmrnhb.com^
||forlumineontor.com^
||gaijiglo.net^
||giwzywnxnk.com^
||gompoozu.net^
||greenplasticdua.com^
||juxupsasu.pro^
||kewuruve.xyz^
||pailridarm.com^
||poosoahe.com^
||pushinpage.com^
||qppdqcid.com^
||seizedbraggingdeceptive.com^
||sellerbackstagejolly.com^
||sevenedgesteve.com^
||speciadnessing.pro^
||stuffedprofessional.com^
||taghaugh.com^
||taigrooh.net^
||thispositionw.biz^
||tyboyyli.xyz^
||vidverto.io^
||wholefreshposts.com^
||zosuptosta.pro^
||zxclan.com^

[okiehsch]

https://github.com/uBlockOrigin/uAssets/commit/290f2dc856ce896bfacd7f8a7bfde388dce342a0

Please remember to not removing third-party blocking filters that accompany a badfilter from the unbreak list,

||example.com^$badfilter
||example.com^$3p

They are there for the stated reason in the linked comment. https://github.com/uBlockOrigin/uAssets/issues/1825#issuecomment-377019866

[Yuki2718]

@felix-22 Done

[Yuki2718]

Removed, thx.

[Yuki2718]

@ryanbr @Khrin ||xtrackme.com^ and ||smac-ad.com^ block access to their sites, they should be with 3p.

[ghajini]

Cleanup

||7cc70.com^
||c44wergiu87heghoconutdx.com^
||conjectwatson.com^
||d23xhr62nxa8qo.cloudfront.net^
||deserterstrugglingdistil.com^
||dzhjmp.com^
||eshouloo.net^
||fmvliiinznfso.com^
||fxwseotmavbde.com^
||garysero.xyz^
||gloomseb.net^
||hadsimz.com^
||king3rsc7ol9e3ge.com^
||las4srv.com^
||linkedgraceless.com^
||lzrikate.com^
||m45assachucsh9w7etts.com^
||mcpuwpush.com^
||militanttrial.com^
||n12qga4ld3zk.com^
||ndbspfpgynbt.com^
||neupwvhayjm.com^
||nonegrysu.pro^
||nxwipuoybxc.com^
||odhrnejegx.com^
||pleastindustress.xyz^
||psauwush.net^
||rategeisteryc.xyz^
||rbyewcokbqlc.com^
||rccsekuitas.com^
||ta3nfsordd.com^
||thatsouh.com^
||untvwqvrk.com^
||vedepmhmdeanoh.xyz^
||whsrdsweq.com^
||wnccxcgvkuoitmy.com^
||worldtraffic.trade^
||wpwocgjsqselo.xyz^
||wwlipxritaoo.com^
||yulunanews.name^

familyporn.tv##.side_spot
flv2mp3.by##.content-right-bar
flv2mp3.by##.horizontal-area
fpo.xxx##.footer-margin
geekzone.co.nz##.adsbygoogle
ladbible.com##.css-1dgm0zi-Advert
lightnovelpub.com##.adsbox
milf300.com#@#.ads-link
msn.com##.intra-article-ad
pornicom.com##.spot-after
truetrophies.com###nn_bfa_wrapper
vstreamhub.com###banner
youx.xxx##.banner
||tubxporn.xxx/js/xp.js

[ghajini]

Cleanup

1] Ublock filters

##[href^="//mage98rquewz.com/"]
pervclips.com###under-video
porntrex.com###exclusive-link
rat.xxx##.spots-title
trueachievements.com##.nn_player_w
||aliascagesboxer.com^
||animeland.tv/ios.html
||antalithate.website^
||asemskull.com^
||attentioniau.com^
||ayahucapasso.website^
||binsaiwo.net^
||blamedbuckie.fun^
||cammpaign.com^
||cirsepuh.net^
||cylize.com^
||d3bbyfw7v2aifi.cloudfront.net^
||dotcom10.info^
||drda5yf9kgz5p.cloudfront.net^
||edgar2al2larngpoer.com^
||ftigholm.site^
||gameads.io^
||goscsbefotbxy.com^
||inancukan.xyz^
||jaxetifu.xyz^
||jswfgfmmuvkuh.com^
||jupied.com^
||knifecontributionproposition.com^
||latentcreeper.com^
||lickingimprovementpropulsion.com^
||mage98rquewz.com^
||methidpossum.com^
||mp3pro.xyz^
||mykadule.xyz^
||myzuphaby.pro^
||napsiguk.com^
||nlyeoehgdhhe.com^
||nndkukagmos.com^
||nvmjdpbjgrh.com^
||nvsviaujqqe.com^
||onpluslean.com^
||ouroborosx.com^
||paddedhayssag.com^
||pertyvaluati.com^
||qofowyki.xyz^
||rajabets.xyz^
||rapolok.com^
||sduptpppm.com^
||simplewebanalysis.com^
||sosigninggrudge.com^
||splaim.com^
||tbadseiepqrl.xyz^
||ugewvoyijwy.com^
||umidqysrw.com^
||uttclimber.com^
||vhjuoaoetdbxt.com^
||vrqsvrnjrhdmek.com^
||wwowww.xyz^
||xhr0.xyz^
||zeyappland.com^
||zhtlvag.com^
||zylytavo.com^

2. ublock unbreak

   
   ! https://github.com/AdguardTeam/AdguardFilters/issues/71177
   javbix.com#@#.adsleft

! javynow.com Hot Videos hidden javynow.com#@#.videos-ad

3. this website `mp3offlineorg` is gone, so

! https://forums.lanik.us/viewtopic.php?p=155744#p155744 ! https://github.com/uBlockOrigin/uAssets/issues/8703 mp3offline.org##+js(nowoif)

[okiehsch]

Thx.

[ghajini]

a few one in

1) ublock filters wrt easylist

pornwhite.com##.content-aside
||affluentshinymulticultural.com^
||althef.com^
||chrif8kdstie.com^
||d1nubxdgom3wqt.cloudfront.net^
||dinterperson.xyz^
||elil.cc/pdev/
||elil.cc/reqe.js
||forecastsinglegait.com^
||housewifehaunted.com^
||ionakasulba.xyz^
||okytgxmxonxim.com^
||pertyvaluati.com^
||sentimenthypocrisy.com^
||separatecolonist.com^
||xjzcbddnj.com^

2) ublock unbreak wrt easylist

! https://github.com/easylist/easylist/pull/9904`
@@||clj.valuecommerce.com/*/vcushion.min.js

! https://github.com/uBlockOrigin/uAssets/issues/13360
thelincolnite.co.uk#@#.inline-ad

3) ublock badware risks wrt easylist

||1redirb.com^
||demnan.com^
||movementsspentlist.com^
||ouhastay.net^
||ponk.pro^

[mapx-]

https://github.com/uBlockOrigin/uAssets/commit/772b5f9c1825b34edf2fa462f664ebed9c4caef1