essandess / adblock2privoxy

Convert adblock config files to privoxy format
https://hackage.haskell.org/package/adblock2privoxy
GNU General Public License v3.0
93 stars 16 forks source link

privoxy crash due to bad filter combination #7

Closed wmyrda closed 5 years ago

wmyrda commented 6 years ago

Running command adblock2privoxy -p /tmp/privoxy -w /tmp/privoxy -d 127.0.0.1:8445 -t privoxy_ab2b.task \ https://easylist-downloads.adblockplus.org/easylist.txt even with only one source creates among others ab2p.system.action file. That file includes:

#workaround for gzip handling privoxy bugs
{ +prevent-compression }
 *.yahoo.com
 *.amazon.com

This results in privoxy crash if config files created got overridden on running app or fails to start if it was not previously running

2018-05-20 16:23:31.825 7f5219ffb700 Error: No server or forwarder response received on socket 9.
2018-05-20 16:59:15.630 7f522c1150c0 Info: Loading filter file: /etc/privoxy/ab2p.filter
2018-05-20 16:59:15.767 7f522c1150c0 Error: Adding re_filter job 's@^referer(?=:[^\n]*[./](?:171gifs\.com|1proxy\.de|2ddl\.download|300mbfilms\.org|300mbmovies4u\.lol|321jav\.com|353online\.com|4horlover\.blogspot\.com|4horlover2\.blogspot\.com|4proxy\.de|61tube\.com|69sugar\.com|6jav\.com|6teentube\.am|7starhd\.com|9xmovies\.site|abcmalayalam\.co|abgdianci\.com|adultdouga\.biz|aflamfree\.net|aflamtorrent\.com|agarios\.org|ahlamtv\.com|al\.ly|alantv\.net|alivefoot\.us|alivefootballstreaming\.com|allpeliculas\.com|alltube\.tv|ally\.sh|amabitch\.com|amateur\.ug|ancensored\.com|andrija-i-andjelka\.com|animakai\.info|anime-music\.info|anime-shitai\.tv|anime-sugoi\.com|animeado\.net|animeai\.org|animeai2\.net|animeid\.io|animelek\.com|animemusicdownload\.info|animepahe\.com|animesenzalimiti\.com|animesonline2hd\.org|animesonlinetk\.info|animesorion\.org|animezone\.pl|anitube\.es|antenasport\.eu|anyanime\.com|apklover\.net|aquariumgays\.com|arab-moviez\.org|arabloads\.net|arabp2p\.com|archived\.moe|artgifsocean\.com|asianporndistrict\.com|asianxv\.com|assistirfilmeshd\.org|assistirfilmesonline2\.net|avonline\.tv|avpockiehd\.com|axxomovies\.in|azkempire\.com|aznude\.com|baixarsomusica\.com|bajarjuegospcgratis\.com|bakacan\.com|balkandownload\.org|balkanje\.com|bdmusicboss\.net|bdsmporn\.us|bdsmstreak\.com|beautiesbondage\.com|becekin\.net|beelink\.in|behchala\.com|bersek\.xyz|bestsongspk\.com|big4umovies\.net|bilasport\.pw|bitch-show\.com|bitporno\.com|blackboxrepack\.com|blacklionmusic\.com|blogqpot\.com|bludv\.com|bokep2017\.com|bokepcewek\.net|bokepseks\.co|bolly2tolly\.com|bouncebreak\.com|brazzershd\.co|btdb\.in|bugiltelanjang17\.com|bypassed\.cab|bypassed\.plus|bypassed\.team|calcioitalia\.stream|camrouge\.com|camwhores\.co|cartoonhd\.be|cartoonhd\.cc|cartoonhd\.globa|cartoonhd\.global|cartoonth12\.com|catchcoin\.pw|catosports\.ml|centraldeanimes\.biz|cholotubex\.com|cinemamkv\.xyz|cinetux\.net|clik\.pw|cliphayho\.com|cloudy\.ec|coastalhut\.com|columbia-xxx\.com|comicporno\.org|comicsmanics\.com|cookiesnetfl1x\.com|cooltamil\.com|coroas40\.com|coshurl\.co|couchtuner\.fr|couchtuner\.nu|cricbox\.net|cwtube\.dj|czechmoneyteens\.com|dailyuploads\.net|dato\.porn|datpiff\.biz|dblatino\.com|dclinks\.info|dd-books\.com|debrideco\.com|demonoid\.co|depedlps\.blogspot\.com|desixnxx\.n' to filter ab2p-unblock-elem-R2456 failed: (pcrs:) Syntax error while parsing command
2018-05-20 16:59:16.416 7f522c1150c0 Info: Loading filter file: /etc/privoxy/ab2p.filter
2018-05-20 16:59:16.648 7f522c1150c0 Info: Loading actions file: /etc/privoxy/ab2p.action
2018-05-20 16:59:17.033 7f522c1150c0 Info: Loading actions file: /etc/privoxy/ab2p.action
2018-05-20 16:59:36.575 7f522c1150c0 Info: Loading filter file: /etc/privoxy/ab2p.system.filter
2018-05-20 16:59:36.575 7f522c1150c0 Info: Loading actions file: /etc/privoxy/ab2p.system.action
2018-05-20 16:59:36.575 7f522c1150c0 Error: error compiling *.yahoo.com from *.yahoo.com: ? * + invalid at offset 0     
2018-05-20 16:59:36.575 7f522c1150c0 Fatal error: can't load actions file '/etc/privoxy/ab2p.system.action': line 23: cannot create URL or TAG pattern from: *.yahoo.com

Removing leading * from those two rules makes privoxy work again

#workaround for gzip handling privoxy bugs
{ +prevent-compression }
 .yahoo.com
 .amazon.com

Seems that particular record is not related to any adblock.txt list used, but is created by adblock2privoxy itself.

wmyrda commented 6 years ago

As a workaround one may use sed -i -e 's:*.:.:' /etc/privoxy/ab2p.system.action after running adblock2privoxy to automatically fix offending lines.

wmyrda commented 6 years ago

@essandess

For pernament fix please add following fix to the code. It patches template resposible for the problem.

diff -Naur adblock2privoxy-9999.old/adblock2privoxy/templates/ab2p.system.action adblock2privoxy-9999/adblock2privoxy/templates/ab2p.system.action
--- adblock2privoxy-9999.old/adblock2privoxy/templates/ab2p.system.action       2018-06-03 11:22:27.374247091 +0200
+++ adblock2privoxy-9999/adblock2privoxy/templates/ab2p.system.action   2018-06-03 11:23:10.974250166 +0200
@@ -28,5 +28,5 @@

 #workaround for gzip handling privoxy bugs
 { +prevent-compression }
- *.yahoo.com
- *.amazon.com
+ .yahoo.com
+ .amazon.com
essandess commented 6 years ago

Note to self: fix this directly in the file ab2p.system.action.

@wmyrda I see leading spaces in this file, which I believe is incorrect privoxy syntax. Have you gone back and checked the privoxy dox to make sure removing the leading * won’t lead to side effects? If so, do you mind tossing in a link?

I’ll come back and address this as soon as I have a moment to do so and check it.

wmyrda commented 6 years ago

Just checked for it and removing space does not help after bringing back * privoxy 3.0.26 still crashes

wmyrda commented 6 years ago

Only side effect of not having { +prevent-compression } is that particular site would not be filtered by privoxy. I do not think that this section belongs to the file which would be regenerated every time a2p is used at all. Those entries have to be manually generated anyways as they could not be derived from any adblock list. Personally I use file to which I add hosts on which I stumble upon and where compression is undesired.

I recommend to remove it completely and add it to separate file. My current example below

diff -Naur adblock2privoxy-9999.old/adblock2privoxy/templates/ab2p.system.action adblock2privoxy-9999/adblock2privoxy/templates/ab2p.system.action
--- adblock2privoxy-9999.old/adblock2privoxy/templates/ab2p.system.action       2018-08-19 09:18:53.334664654 +0200
+++ adblock2privoxy-9999/adblock2privoxy/templates/ab2p.system.action   2018-08-19 09:30:40.406010437 +0200
@@ -25,8 +28,3 @@

 {-block}
 TAG:^ab2p-unblock-s$
-
-#workaround for gzip handling privoxy bugs
-{ +prevent-compression }
- *.yahoo.com
- *.amazon.com

gzipfix.action -- add actionsfile gzipfix.action to privoxy's config file to use it

# Workaround for gzip handling privoxy bugs
{ +prevent-compression }
#Arabic
\.alalam\.ir
\.muraselon\.com

#English
\.almasdarnews\.com
\.amazon\.com
\.belden\.com
\.cyberciti\.biz
\.defence24\.com
^electronicintifada\.net
\.eurasiafuture\.com
\.europa\.eu
\.foreignpolicyjournal\.com
\.globalresearch\.ca
\.kbb\.com
\.kodi\.tv
^linoxide\.com
^linuxconfig\.org
\.mintpressnews\.com
\.opensubtitles\.org
\.phoronix\.com
\.pikcat\.com
\.pictasite\.com
\.pointdeveloper\.com
\.prohavit\.com
\.russia-insider\.com
\.skepticalraptor\.com
\.sott\.net
\.targus\.com
^tune\.pk
\.yahoo\.com
\.youtube\.com
\.veteranstoday\.com

#Polish
\.autocentrum\.pl
^bezux\.pl
^businessinsider\.com\.pl
\.centrumdruku\.com\.pl
\.centrumpapieru\.pl
^chamsko\.pl
\.chip\.pl
^ciekawe\.org
\.computerworld\.pl
\.computerworld\.pl
\.defence24\.pl
^demotywatory\.pl
^dobry-kon\.pl
^dorzeczy\.pl   #still does not filter
\.domiporta\.pl
\.download\.net\.pl
^echaswantewita\.blogspot\.com  #still does not filter
\.fabrykawafelkow\.pl
\.fangol\.pl
\.filmweb\.pl
^fortnitepolska\.pl
\.homify\.pl
\.hotfrog\.pl
\.hulahop\.pl
^kariera\.pl
\.komputerswiat\.pl
\.lidl\.pl
\.lublin112\.pl
\.idg\.pl
^innemedium\.pl
^inwestycje-rzeszow\.pl
^jejportal\.pl
\.jobisjob\.pl
\.makehomeeasier\.pl
\.mamissima\.pl
\.media2\.pl
\.miniminiplus\.pl      #still does not filter
\.meble\.pl
^napisy24\.pl
^nf\.pl         #still does not filter
^ototorun\.pl
^oko\.press
^paczaizm\.pl
\.pch24\.pl
\.pcworld\.pl
^plonszczak\.pl
^podkarpackisport\.pl
\.programosy\.pl
^pupilo\.pl
\.purepc\.pl
\.seat-auto\.pl
^selsey\.pl
^silesion\.pl
^socialpress\.pl
\.sosrodzice\.pl
^specjalski\.pl
^stooq\.pl
^tojuzbylo\.pl
^tosport\.pl
^tylkoastronomia\.pl    #still does not filter
^tylkomedycyna\.pl
^tylkonauka\.pl
^superdania\.pl
^media2\.pl
^mszana-dolna\.eu
^niezlomni\.com
\.weatheronline\.pl
^webinsider\.pl
^wolna-polska\.pl
\.wp\.pl
^wsensie\.pl
www\.tvn24\.pl
^zmianynaziemi\.pl

#Russian
.gazeta\.ru
essandess commented 5 years ago

Addressed in https://github.com/essandess/adblock2privoxy/commit/cb63aad374e94adab5acc8a26cc8158011202875.