Adblock decoder ignore some portion when decoding

funilrys commented 6 years ago

As reported by @dnmTX at https://github.com/Ultimate-Hosts-Blacklist/dev-center/issues/9:

everything with ##[href^=....

are ignored.

keczuppp commented 3 years ago

OT response to spirillen

I don't get your problem, GH does highlight the changes, just like on your screenshoot, with green color: ![capture_001_11-03-2021_10-32-05](https://user-images.githubusercontent.com/74409207/110767113-2c7b2b80-8256-11eb-9acc-7e10dd6270c1.png) ![capture_002_11-03-2021_10-32-27](https://user-images.githubusercontent.com/74409207/110767123-2f761c00-8256-11eb-86e5-0d6aa090344d.png) ![capture_003_11-03-2021_10-32-37](https://user-images.githubusercontent.com/74409207/110767129-313fdf80-8256-11eb-81c4-4c39de1637bd.png)

funilrys commented 3 years ago

Mhh, it's true. It was confusing to introduce the --aggressive argument. But as the time it was first introduced, I wasn't even sure that that thing that I engineered will be actually used.

My objective was to try to be as accurate as possible and to reduce false positives when people are using the output of PyFunceble directly into their workflow ...

The AdBlock decoder itself is self-engineered. So the more input I will get the better it will. I have literally no way to imagine everything.

I'm currently working on v4.0.0bNand there is in my opinion - after analysis - no real reason to split everything anymore. This tool should decode as much as possible.

Therefore, I'm willing to change the direction: What about PyFunceble trying to code as much as possible - if not all.

Inputs from users are highly welcome because I'm not actively writing blocklists.

@keczuppp thank you for your table which I will use for the tests.

Is this new direction fair enough (for everyone)?

cc @kulfoon @spirillen @dnmTX @

spirillen commented 3 years ago

OT to @keczuppp

I most clicked on a wrong "version"... ![image](https://user-images.githubusercontent.com/44526987/110859438-b6111480-82bb-11eb-83d8-35580c212727.png)

spirillen commented 3 years ago

Moved my answer to https://github.com/funilrys/PyFunceble/issues/227 as It's OT to OP's post and I hope there will be more activities in replies to this topic Therefore, I'm willing to change the direction:

keczuppp commented 3 years ago

(my reply is also a reply to https://github.com/funilrys/PyFunceble/issues/227#issuecomment-797239112 at the same time):

yeah, to create a good quality Adblock Decoder is not an easy task, it's tricky, it's a good challenge in programming skills, to extract good domains and avoid false positives at the same time, if you don't feel up to it or you think it's too complicated and not worth to continue the developement, you can replace it with an easier and simpler "decode everything" Adblock Decoder mode, but it's not a magic potion to solve the Adblock Decoder problem, such mode finally will not be better because it will give too many useless false positives which will clutter the output list, there seem to be no easy way to go (no "magic bullet")...
the default "HOSTS" mode should stay, because it's limited to parsing only a few kind of HOSTS compatible filters and that should not cause any problems, it's easy to implement and very useful for hosts USERS as a great supplement for their HOSTS lists, if anything cause problems in this mode, just limit extraction to what is listed here https://github.com/funilrys/PyFunceble/issues/13#issuecomment-619415365, but it seems you extract way too much in this mode on your own and it might cause troubles...

funilrys commented 3 years ago

Please take my commit and the underlying tests as the response. Is it still too much @keczuppp ?

Let's discuss the future of that specific decoder. I'll inject any future report about missing decoding into the tests. So the more reports, the better that decoder will be 😄

As I wrote, I'm not one of those who write a filter list... So help or directions are welcome!

keczuppp commented 3 years ago

Hello, I was already trying to test the new version of Adblock Decoder (4.0.0b35) but:

the standalone Adblock Decoder https://github.com/PyFunceble/adblock-decoder hasn't been updated (I was using it for testing before)
when I was trying to run it from PyFunceble I was unable coz I received some errors in Python (3.7.6):

error

 Finished processing dependencies for PyFunceble-dev==4.0.0b35

D:\download_big_temp\_koding\PyFunceble-dev>pyfunceble
Traceback (most recent call last):
  File "D:\download_big_temp\_koding\Python37\Scripts\pyfunceble-script.py", line 33, in <module>
    sys.exit(load_entry_point('PyFunceble-dev==4.0.0b35', 'console_scripts', 'pyfunceble')())
  File "D:\download_big_temp\_koding\Python37\lib\site-packages\pyfunceble_dev-4.0.0b35-py3.7.egg\Py
Funceble\cli\entry_points\pyfunceble\cli.py", line 1022, in tool
  File "D:\download_big_temp\_koding\Python37\lib\site-packages\pyfunceble_dev-4.0.0b35-py3.7.egg\Py
Funceble\config\loader.py", line 370, in start
  File "D:\download_big_temp\_koding\Python37\lib\site-packages\pyfunceble_dev-4.0.0b35-py3.7.egg\Py
Funceble\config\loader.py", line 331, in get_config_file_content
  File "D:\download_big_temp\_koding\Python37\lib\site-packages\pyfunceble_dev-4.0.0b35-py3.7.egg\Py
Funceble\helpers\dict.py", line 290, in from_yaml_file
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\user\\AppData\\Local\\Temp\\tmp_
oox9mr2'

also even if somehow you help me to fix Python errors, I still don't know how to use it from PyFunceble without doing any DNS queries etc (how to use it just like the standalone decoder: just to put an input file and get an output file), as I want to put some fitler list to test and decode it but I'm not interested spending hours waiting until it finish useless DNS queries (they are useless and garbare in case of testing the Adblock Decoder), was unable to find any info about skipping DNS queries in Adblock Decoder mode, in the documentation.

funilrys commented 3 years ago

@keczuppp Thanks for the notice. I'll update the AdBlock decoder project as soon as possible.

The simple way, is the pyfunceble --syntax --adblock --aggressive -f [file] arguments. 😄

Note to self: Cleanup documentation.

keczuppp commented 3 years ago

So I've tried the newest version v4.0.0b36. and:

PyFuncable has worked finally, it doesn't throw the error I mentioned in my previous comment anymore ( FileNotFoundError: [Errno 2] No such file or directory: ) so at least I was able to run the whole PyFunceble
however, when I ran the Adblock Decoder with the command you provided in your comment, with a few lists (EasyList and also two main polish ads filter lists), it gets broken and throws another errors:
- with https://easylist.to/easylist/easylist.txt - see Errors 1 log spoiler - it analysed nothing, crashed at the beginning
- with https://easylist-downloads.adblockplus.org/easylistpolish.txt - see Errors 2 log spoiler - it analysed some domains and then crashed
- with https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-adblock-filters/adblock_ublock.txt - see Errors 3 log spoiler - it analysed nothing, crashed at the beginning

Errors 1 log (EasyList)

``` D:\download_big_temp\_koding>pyfunceble --syntax --adblock --aggressive -f easylist.txt ######## ## ## ######## ## ## ## ## ###### ######## ######## ## ######## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## #### ## ## ## #### ## ## ## ## ## ## ## ######## ## ###### ## ## ## ## ## ## ###### ######## ## ###### ## ## ## ## ## ## #### ## ## ## ## ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## ####### ## ## ###### ######## ######## ######## ######## You are using the Beta version of PyFunceble 4.0.0! Please take the time to communicate with us when you notice something unusual. Fatal Error: 'bool' object has no attribute 'replace' Traceback (most recent call last): File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 864, in start self.fill_to_test_queue_from_protocol() File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 593, in fill_to_test_queue_from_protocol handle_file(protocol) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 533, in handle_file cidr2subject=self.cidr2subject, File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\utils\testing.py", li ne 228, in get_subjects_from_line .set_data_to_convert(line) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 429, in get_converted result.update(self._decode_v5(self.data_to_convert)) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 382, in _decode_v5 result.update(self._decode_options(options.split(","))) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 211, in _decode_options result.add(self.extract_base(matched)) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 156, in extract_base subject = subject.replace("*", "").replace("~", "") AttributeError: 'bool' object has no attribute 'replace' Process pyfunceble_tester_worker_2: Process pyfunceble_tester_worker_1: Traceback (most recent call last): File "d:\download_big_temp\_koding\python37\lib\multiprocessing\connection.py", line 312, in _recv _bytes nread, err = ov.GetOverlappedResult(True) BrokenPipeError: [WinError 109] Potok został zakończony During handling of the above exception, another exception occurred: Traceback (most recent call last): File "d:\download_big_temp\_koding\python37\lib\multiprocessing\process.py", line 297, in _bootstr ap self.run() D:\download_big_temp\_koding> ```

Errors 2 log (EasyList Polish)

``` D:\download_big_temp\_koding>pyfunceble --syntax --adblock --aggressive -f easylistpolish.txt ######## ## ## ######## ## ## ## ## ###### ######## ######## ## ######## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## #### ## ## ## #### ## ## ## ## ## ## ## ######## ## ###### ## ## ## ## ## ## ###### ######## ## ###### ## ## ## ## ## ## #### ## ## ## ## ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## ####### ## ## ###### ######## ######## ######## ######## You are using the Beta version of PyFunceble 4.0.0! Please take the time to communicate with us when you notice something unusual. Subject Status Source ---------------------------------------------------------------------------------------------------- ----------- ---------- ←[30m←[42mczasdzieci.pl VALID SYNTAX ←[30m←[42mwlbetathome.adsrv.eacdn.com VALID SYNTAX ←[30m←[42mmbank.pl VALID SYNTAX ←[30m←[42mrtb.4finance.com VALID SYNTAX ←[30m←[42mapp.freshmail.com VALID SYNTAX ←[30m←[42mmandarinodesign.eu VALID SYNTAX ←[30m←[42mad-work.pl VALID SYNTAX ←[30m←[42mwirtualnyregion.pl VALID SYNTAX ←[30m←[42madsearch.pl VALID SYNTAX ←[30m←[42maffiliates-solutions.com VALID SYNTAX ←[30m←[42madsnet.pl VALID SYNTAX ←[30m←[42mbanmax.com VALID SYNTAX ←[30m←[42mconverti.se VALID SYNTAX ←[30m←[42mconvertiser.com VALID SYNTAX ←[30m←[42mhub.com.pl VALID SYNTAX ←[30m←[42mincontext.pl VALID SYNTAX ←[30m←[42mleadstar.pl VALID SYNTAX ←[30m←[42mmedia.stsaff.pl VALID SYNTAX ←[30m←[42mnetsalesmedia.pl VALID SYNTAX ←[30m←[42mpocketads.pl VALID SYNTAX ←[30m←[42mreklamawadowice24.pl VALID SYNTAX ←[30m←[42mrewords.pl VALID SYNTAX ←[30m←[42mrenormaliseras.xyz VALID SYNTAX ←[30m←[42mshopeneo.network VALID SYNTAX ←[30m←[42mspacead.pl VALID SYNTAX ←[30m←[42mtvtoss.com VALID SYNTAX ←[30m←[42mwaytogrow.pl VALID SYNTAX ←[30m←[42mad.admitad.com VALID SYNTAX ←[30m←[42mad.e-lider.pl VALID SYNTAX ←[30m←[42mad.eko-7.com.pl VALID SYNTAX ←[30m←[42maff.bstatic.com VALID SYNTAX ←[30m←[42mbusinessclick.biz.pl VALID SYNTAX ←[30m←[42mavanti.fashion VALID SYNTAX ←[30m←[42mcdn.leadbit.com VALID SYNTAX ←[30m←[42mcomperialead.pl VALID SYNTAX ←[30m←[42mcontexthub.net VALID SYNTAX ←[30m←[42mczasdzieci.home.pl VALID SYNTAX ←[30m←[42mec.hub2.com.pl VALID SYNTAX ←[30m←[42meuphoniserent.xyz VALID SYNTAX ←[30m←[42mlnaff.pl VALID SYNTAX ←[30m←[42moffersprovider.widget.onet.pl VALID SYNTAX ←[30m←[42mpartnerzyapi.ceneo.pl VALID SYNTAX ←[30m←[42mppwidget.skapiec.pl VALID SYNTAX ←[30m←[42mqwerty1.co.pl VALID SYNTAX ←[30m←[42mr.pless.nazwa.pl VALID SYNTAX ←[30m←[42mreklamy.hostings.pl VALID SYNTAX ←[30m←[42msmartclick.pl VALID SYNTAX ←[30m←[42msolutions4ad.com VALID SYNTAX ←[30m←[42mstatic.travelist.pl VALID SYNTAX ←[30m←[42msystem.mondeos.pl VALID SYNTAX ←[30m←[42mthc-thc.com VALID SYNTAX ←[30m←[42mtmefekt.pl VALID SYNTAX ←[30m←[42mwydawca.lead.network VALID SYNTAX ←[30m←[42mpopups.afftrack001.com VALID SYNTAX ←[30m←[42m24gliwice.pl VALID SYNTAX ←[30m←[42m24opole.pl VALID SYNTAX ←[30m←[42mfilmweb.com VALID SYNTAX ←[30m←[42m40ton.net VALID SYNTAX ←[30m←[42m7dni.com.pl VALID SYNTAX ←[30m←[42mad.polskiprzemysl.com.pl VALID SYNTAX ←[30m←[42mad.prv.pl VALID SYNTAX ←[30m←[42m300polityka.pl VALID SYNTAX ←[30m←[42madform.net VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maferyprawa.eu VALID SYNTAX ←[30m←[42maktyw14.net VALID SYNTAX ←[30m←[42maktyw14.net VALID SYNTAX ←[30m←[42maktyw14.net VALID SYNTAX ←[30m←[42malicdn.com VALID SYNTAX ←[30m←[42mtelchina.pl VALID SYNTAX ←[30m←[42mallebiznes.pl VALID SYNTAX ←[30m←[42mwlodawa.net VALID SYNTAX ←[30m←[42mandroidpolska.pl VALID SYNTAX ←[30m←[42mangielskieespresso.pl VALID SYNTAX ←[30m←[42mbelekaj.eu VALID SYNTAX ←[30m←[42mapp.travellead.pl VALID SYNTAX ←[30m←[42marcheton.pl VALID SYNTAX ←[30m←[42marpass.nazwa.pl VALID SYNTAX ←[30m←[42matthost.pl VALID SYNTAX ←[30m←[42mwarownie.pl VALID SYNTAX ←[30m←[42maudiostereo.pl VALID SYNTAX ←[30m←[42mautoline.com.pl VALID SYNTAX ←[30m←[42mautomotivesuppliers.pl VALID SYNTAX ←[30m←[42mautomotivesuppliers.pl VALID SYNTAX ←[30m←[42mautorak.com.pl VALID SYNTAX ←[30m←[42mb24tv.pl VALID SYNTAX ←[30m←[42mnadwisla24.pl VALID SYNTAX ←[30m←[42mb24tv.pl VALID SYNTAX ←[30m←[42mbaby-shower.pl VALID SYNTAX ←[30m←[42mbankier.pl VALID SYNTAX ←[30m←[42mziemiakepinska.pl VALID SYNTAX ←[30m←[42mbatuu.pl VALID SYNTAX ←[30m←[42mwizaz.pl VALID SYNTAX ←[30m←[42mbaxu.pl VALID SYNTAX ←[30m←[42mbeerpubs.pl VALID SYNTAX ←[30m←[42mbetonline.net.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbezale.pl VALID SYNTAX ←[30m←[42mbielskiedrogi.pl VALID SYNTAX ←[30m←[42mbielskiedrogi.pl VALID SYNTAX ←[30m←[42mbielskiedrogi.pl VALID SYNTAX ←[30m←[42mbiotechnologia.pl VALID SYNTAX ←[30m←[42mbitcoin-online.pl VALID SYNTAX ←[30m←[42mbithub.pl VALID SYNTAX ←[30m←[42mblendy.pl VALID SYNTAX ←[30m←[42mblogomaniak.pl VALID SYNTAX ←[30m←[42mblogprezesa.pl VALID SYNTAX ←[30m←[42mblogspot.com VALID SYNTAX ←[30m←[42mmistrzbranzy.pl VALID SYNTAX ←[30m←[42mbobrowniki.tv VALID SYNTAX ←[30m←[42mbobrowniki.tv VALID SYNTAX ←[30m←[42mbokser.org VALID SYNTAX ←[30m←[42mbolec.info VALID SYNTAX ←[30m←[42mbooking.com VALID SYNTAX ←[30m←[42mkazimierzdolny24.pl VALID SYNTAX ←[30m←[42mbronradom.pl VALID SYNTAX ←[30m←[42mbronradom.pl VALID SYNTAX ←[30m←[42mbronradom.pl VALID SYNTAX ←[30m←[42mbronradom.pl VALID SYNTAX ←[30m←[42mbronradom.pl VALID SYNTAX ←[30m←[42mbstok.pl VALID SYNTAX ←[30m←[42mburdadigital.pl VALID SYNTAX ←[30m←[42mfocus.pl VALID SYNTAX ←[30m←[42mbusiarze.com.pl VALID SYNTAX ←[30m←[42mbusiarze.com.pl VALID SYNTAX ←[30m←[42mbytomski.pl VALID SYNTAX ←[30m←[42mc.spolecznosci.net VALID SYNTAX ←[30m←[42mdlastudenta.pl VALID SYNTAX ←[30m←[42mcba.pl VALID SYNTAX ←[30m←[42mcdn-lubimyczytac.pl VALID SYNTAX ←[30m←[42mlubimyczytac.pl VALID SYNTAX ←[30m←[42mcdn.dcsaas.net VALID SYNTAX ←[30m←[42msklepbazant.pl VALID SYNTAX ←[30m←[42mswiatmodeli.eu VALID SYNTAX ←[30m←[42marchigame.pl VALID SYNTAX ←[30m←[42mszczecinek.com VALID SYNTAX ←[30m←[42mceneo.pl VALID SYNTAX ←[30m←[42mexerim.pl VALID SYNTAX ←[30m←[42mkomputery-pc.info VALID SYNTAX ←[30m←[42mpmi24.info VALID SYNTAX ←[30m←[42mpch24.info VALID SYNTAX ←[30m←[42mnowinylokalne.pl VALID SYNTAX ←[30m←[42mpgo24.pl VALID SYNTAX ←[30m←[42mppw.fishing VALID SYNTAX ←[30m←[42mtromil.pl VALID SYNTAX ←[30m←[42mgrojec24.net VALID SYNTAX ←[30m←[42mcentrumkultury.eu VALID SYNTAX ←[30m←[42mcentrumkultury.eu VALID SYNTAX ←[30m←[42mceny-zlomu.pl VALID SYNTAX ←[30m←[42mchemiabudowlana.info VALID SYNTAX ←[30m←[42mceny-zlomu.pl VALID SYNTAX ←[30m←[42mchemiaibiznes.com.pl VALID SYNTAX ←[30m←[42mchlodnictwoiklimatyzacja.pl VALID SYNTAX ←[30m←[42mciechanowinaczej.pl VALID SYNTAX ←[30m←[42mciechanowinaczej.pl VALID SYNTAX ←[30m←[42mcmas.pl VALID SYNTAX ←[30m←[42mcn-tryton.pl VALID SYNTAX ←[30m←[42mcodziennikmlawski.pl VALID SYNTAX ←[30m←[42mcodziennikmlawski.pl VALID SYNTAX ←[30m←[42mcodziennikmlawski.pl VALID SYNTAX ←[30m←[42mcontentstream.pl VALID SYNTAX ←[30m←[42mshareinfo.pl VALID SYNTAX ←[30m←[42mcowwilanowie.pl VALID SYNTAX ←[30m←[42mczestochowskie24.pl VALID SYNTAX ←[30m←[42mcyfrowaekonomia.pl VALID SYNTAX ←[30m←[42mczestochowskie24.pl VALID SYNTAX ←[30m←[42mdentoforum.pl VALID SYNTAX ←[30m←[42mdi.com.pl VALID SYNTAX ←[30m←[42mdi.com.pl VALID SYNTAX ←[30m←[42mdirect.money.pl VALID SYNTAX ←[30m←[42mdobrewiadomosci.eu VALID SYNTAX ←[30m←[42mdodajauto.pl VALID SYNTAX ←[30m←[42mdodajauto.pl VALID SYNTAX ←[30m←[42mdogosfera.pl VALID SYNTAX ←[30m←[42mdogosfera.pl VALID SYNTAX ←[30m←[42mdomenergo.com VALID SYNTAX ←[30m←[42mdopilar.pl VALID SYNTAX ←[30m←[42mdopilar.pl VALID SYNTAX ←[30m←[42mdrzewkozabutelke.pl VALID SYNTAX ←[30m←[42mdx-team.org VALID SYNTAX ←[30m←[42mdynacrems.wp.pl VALID SYNTAX ←[30m←[42mdz-ow.pl VALID SYNTAX ←[30m←[42mdziennikpolski24.pl VALID SYNTAX ←[30m←[42mdziennikzwiazkowy.com VALID SYNTAX ←[30m←[42mdzierzgon-twojemiasto.pl VALID SYNTAX ←[30m←[42mdzisiajwgliwicach.pl VALID SYNTAX ←[30m←[42me-hotelarz.pl VALID SYNTAX ←[30m←[42me-hotelarz.pl VALID SYNTAX ←[30m←[42me-kg.pl VALID SYNTAX ←[30m←[42me-kolo.pl VALID SYNTAX ←[30m←[42me-kolo.pl VALID SYNTAX ←[30m←[42me-petrol.pl VALID SYNTAX ←[30m←[42me-pingpong.pl VALID SYNTAX ←[30m←[42me-pingpong.pl VALID SYNTAX ←[30m←[42me-play.eu VALID SYNTAX ←[30m←[42me-pingpong.pl VALID SYNTAX ←[30m←[42me-play.eu VALID SYNTAX ←[30m←[42me-stargard.pl VALID SYNTAX ←[30m←[42mebarlinek.pl VALID SYNTAX ←[30m←[42mebookpoint.pl VALID SYNTAX ←[30m←[42mswiatczytnikow.pl VALID SYNTAX ←[30m←[42mautofanatyk.pl VALID SYNTAX ←[30m←[42mebroker.pl VALID SYNTAX ←[30m←[42micyfrowypolsat.pl VALID SYNTAX ←[30m←[42mmojeanonse.pl VALID SYNTAX ←[30m←[42mec.bankier.pl VALID SYNTAX ←[30m←[42mmavelo.pl VALID SYNTAX ←[30m←[42mmiedziak.info.pl VALID SYNTAX ←[30m←[42mtutajglogow.pl VALID SYNTAX ←[30m←[42mtutajlegnica.pl VALID SYNTAX ←[30m←[42mtutajpolkowice.pl VALID SYNTAX ←[30m←[42mec.bankier.pl VALID SYNTAX ←[30m←[42micyfrowypolsat.pl VALID SYNTAX ←[30m←[42mechogorzowa.pl VALID SYNTAX ←[30m←[46mechogorzowa.pl^ INVALID SYNTAX ←[30m←[42medunews.pl VALID SYNTAX ←[30m←[42meduson.pl VALID SYNTAX ←[30m←[42mefilmy.tv VALID SYNTAX Fatal Error: 'bool' object has no attribute 'replace' ←[30m←[46meduson.pl^ INVALID SYNTAX Traceback (most recent call last): File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 864, in start self.fill_to_test_queue_from_protocol() File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 593, in fill_to_test_queue_from_protocol handle_file(protocol) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 533, in handle_file cidr2subject=self.cidr2subject, File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\utils\testing.py", li ne 228, in get_subjects_from_line .set_data_to_convert(line) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 429, in get_converted result.update(self._decode_v5(self.data_to_convert)) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 382, in _decode_v5 result.update(self._decode_options(options.split(","))) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 211, in _decode_options result.add(self.extract_base(matched)) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 156, in extract_base subject = subject.replace("*", "").replace("~", "") AttributeError: 'bool' object has no attribute 'replace' ←[30m←[42mekologia.guru VALID SYNTAX ←[30m←[42megorzow.pl VALID SYNTAX ←[30m←[42mekologia.guru VALID SYNTAX ←[30m←[42mekologia.pl VALID SYNTAX ←[30m←[42mekorodzice.pl VALID SYNTAX ←[30m←[42mekstrastats.pl VALID SYNTAX Process pyfunceble_tester_worker_2: Traceback (most recent call last): Process pyfunceble_producer_worker_1: File "d:\download_big_temp\_koding\python37\lib\multiprocessing\process.py", line 297, in _bootstr ap self.run() Process pyfunceble_tester_worker_1: File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\processes\workers\bas e.py", line 434, in run raise exception Traceback (most recent call last): ```

Errors 3 log (Official Polish Filters for AdBlock, uBlock Origin & AdGuard)

``` D:\download_big_temp\_koding>pyfunceble --syntax --adblock --aggressive -f adblock_ublock.txt ######## ## ## ######## ## ## ## ## ###### ######## ######## ## ######## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## #### ## ## ## #### ## ## ## ## ## ## ## ######## ## ###### ## ## ## ## ## ## ###### ######## ## ###### ## ## ## ## ## ## #### ## ## ## ## ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## ####### ## ## ###### ######## ######## ######## ######## You are using the Beta version of PyFunceble 4.0.0! Please take the time to communicate with us when you notice something unusual. Fatal Error: 'bool' object has no attribute 'replace' Traceback (most recent call last): File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 864, in start self.fill_to_test_queue_from_protocol() File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 593, in fill_to_test_queue_from_protocol handle_file(protocol) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\system\launcher.py", line 533, in handle_file cidr2subject=self.cidr2subject, File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\cli\utils\testing.py", li ne 228, in get_subjects_from_line .set_data_to_convert(line) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 429, in get_converted result.update(self._decode_v5(self.data_to_convert)) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 382, in _decode_v5 result.update(self._decode_options(options.split(","))) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 211, in _decode_options result.add(self.extract_base(matched)) File "d:\download_big_temp\_koding\python37\lib\site-packages\PyFunceble\converter\adblock_input_l ine2subject.py", line 156, in extract_base subject = subject.replace("*", "").replace("~", "") AttributeError: 'bool' object has no attribute 'replace' D:\download_big_temp\_koding> ```

funilrys commented 3 years ago

@keczuppp, b37 is available and it should fix the error you reported.

Thanks again for testing !

funilrys commented 3 years ago

@keczuppp, the adblock-decoder is also upgraded to use the 4.0.0bX of PyFunceble.

keczuppp commented 3 years ago

yep, good work:

the bugs are fixed, and PyFunceble embeded Adblock-Decoder is finally working, the standalone version is working as well
the split introduced in the rewrite is good (as it should be since the beginning)
the new decoder fixes all failures from https://github.com/funilrys/PyFunceble/issues/13#issuecomment-749607193, except the last 3
good enhanced manual for the standalone Adblock Decoder https://github.com/funilrys/PyFunceble/issues/13#issuecomment-802227216

more tests later

keczuppp commented 3 years ago

And don't laught at me, fvcktard.

keczuppp commented 3 years ago

strange, I tried today the standalone Adblock Decoder and it crashes and throws errors
also it wasn't crashing yesterday, and I didn't change anything since yesterday, but I'm not sure whether I have been testing the same filter lists yesterday or not, so perhaps the bug was here already yesterday

spoiler errors log


D:\download_big_temp\_koding>adblock2plain --aggressive -o output2.txt easylistpolish.txt
Traceback (most recent call last):
  File "d:\download_big_temp\_koding\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:\download_big_temp\_koding\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\download_big_temp\_koding\Python37\Scripts\adblock2plain.exe\__main__.py", line 7, in <mo
dule>
  File "d:\download_big_temp\_koding\python37\lib\site-packages\adblock_decoder\cli.py", line 104, i
n adblock2plain
    args.input_file, args.aggressive, output=args.output
  File "d:\download_big_temp\_koding\python37\lib\site-packages\adblock_decoder\core\adblock2plain.p
y", line 80, in process_conversion
    for line in self.input:
  File "d:\download_big_temp\_koding\python37\lib\encodings\cp1250.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1350: character maps to <unde
fined>

D:\download_big_temp\_koding>adblock2plain --aggressive -o output2.txt easylist.txt
Traceback (most recent call last):
  File "d:\download_big_temp\_koding\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:\download_big_temp\_koding\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\download_big_temp\_koding\Python37\Scripts\adblock2plain.exe\__main__.py", line 7, in <mo
dule>
  File "d:\download_big_temp\_koding\python37\lib\site-packages\adblock_decoder\cli.py", line 104, i
n adblock2plain
    args.input_file, args.aggressive, output=args.output
  File "d:\download_big_temp\_koding\python37\lib\site-packages\adblock_decoder\core\adblock2plain.p
y", line 80, in process_conversion
    for line in self.input:
  File "d:\download_big_temp\_koding\python37\lib\encodings\cp1250.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x83 in position 5977: character maps to <unde
fined>

the embeded Adblock Decoder doesn't crash when testing same filter lists
the filter lists are just downloaded in Firefox from the links given here: https://github.com/funilrys/PyFunceble/issues/13#issuecomment-801787703 and in Notepad++ it says they are UTF-8 (Without BOM)

funilrys commented 3 years ago

And don't laught at me, fvcktard.

Who is laughing at you?

We are all here for some constructive work, enhancement, and discussion in our free time. I personally take any input I can get regarding the decoder. I don't have time to laugh at someone when they are giving constructive inputs.

If it is because of my emoji, sorry if it offended you. It wasn't meant to harm.

Your last 3 cases are now into the source code that's going to be deployed next.

I'm going to look into the issue of the standalone decoder later.

funilrys commented 3 years ago

@keczuppp Please update and test the adblock-decoder

keczuppp commented 3 years ago

funilrys : I'm going to look into the issue of the standalone decoder later. @keczuppp Please update and test the adblock-decoder

it's fixed now in 1.2.0

funilrys : Your last 3 cases are now into the source code that's going to be deployed next.

it's fixed now in the standalone Adblock Decoder 1.2.0
but in the embeded Adblock Decoder PyFunceble 4.0.0b39 it's fixed at 66% because site21.com is still not being extracted

Also:

I know you have a reason to not do so in the embeded Adblock Decoder, but shouldn't at least the standalone Adblock Decoder remove duplicates and then sort in a-z order? If not internally, then at least in the output. A parameter like --clean could be added.

==================================================

OFF-TOPIC

> funilrys : > Who is laughing at you? > If it is because of my emoji, Yes, it was about "simple way" + the emoji, your comment looks like you wanted to show how stupid I am just because I missed something which you describe as "simple" (a parameter + the fact it should be put in the conjunction with other parameters) in the (big) documentation, which might be not so obvious. Was it more funny to you, than your friend being unable to view history of my comment https://github.com/funilrys/PyFunceble/issues/13#issuecomment-797071985 ? Why didn't you laught at him the same like at me? Oh, because he is your firend, so you can't laught at your friends, just like you did at me. I could laught (by putting a laugh emoji) at him just like you at me, furthermore, I could lught at you (by putting a laugh emoji), every time the Adblock Decoder or PyFunceble crashes making you looking like a fool (but your are a good developer and bugs are normal thing in programming, unavoidable by a human being). And then say: "sorry if it offended you. It wasn't meant to harm." but I'm not a such person. > funilrys : sorry if it offended you. It wasn't meant to harm. Really? Then why didn't you explain what was the purpose of the emoiji in your comment then. Do you want to just tell me you put the emoji for no reason, or just because you were in a happy mood and just by accident it was looking like you were laughting at me...Saying "It wasn't meant to harm you" and at the same time avoiding to explain what was the purpose of the emojii then , seems not too clear. I don't believe your cheap explanation, lie to yourself, I spent much time analysing, whether your intention was to laught at me or not, and something said me very clear you were. Just don't do it again at me, or watch where you put laugh emojis, it's not Facebook, but since they implemented emojis in GitHub it seems it turned into FaceHub, abused by trolls, they abuse emojis to troll other people at every occasion., it's a plague and infection of GitHub. I consider you a positive person and great developer overall, but just don't do it again.

keczuppp commented 3 years ago

As for the domain= filters (for ex. domain=page1.com|page2.com):

the last 3 failures from my previous comment were just example failures, the point is the decoder, instead of fixing particular domain= failures, should extract all domains regardless what is on the left or right side of domain=, this inculdes @@ filters (because in --aggressive mode we should extract all domains)
but currently the decoder extracts about a half of domains, to prove it I copy-pasted all domain= lines from several popular / big adblock filter lists + two polish lists: EasyList, EasyPrivacy, AdGuard Base, AdGuard Tracking Protection, Official Polish Filters for AdBlock, uBlock Origin & AdGuard, EasyList Polish and put into a single file / list (see domain=.zip), which contain about 11425 domains, but Adblock Decoder extracts only about a half (5426 domains)
by the way, the number of false positives is reasonably low, 39 out of 5426 domains :

false hits

&Type=Event.CPT&
-300x250.
-Background-1280x10241.
-spotify-com.akamaized.net
-tag.js
.
.cdn.digitaloceanspaces.com
.cdnjquery.com
.ch
.com
.criteo.com
.criteo.net
.digitaloceanspaces.com
.engageya.com
.filma24.
.gif
.html
.html|
.imagetwist.com
.impact-ad.jp
.jpg
.js
.js|
.m3u8
.min.js
.mp3
.mp4
.mp4.kakaoad.
.mp4|
.netdna-ssl.com
.php
.pl
.pornhub.com
.r.msn.com
.roofandfloor.com
.smithsonian.museum
.ssl-images-amazon.com
.ts
.xml

domain=.zip

spirillen commented 3 years ago

Hey @keczuppp

.. the fact it should be put in the conjunction with other parameters) in the (big) documentation, which might be not so obvious.

What are you missing in the docs? and Could you elaborate it in a issue at https://github.com/spirillen/PyFunceble/issues (The repo I make the docs from)

Was it more funny to you, than your friend being unable to view history of my comment #13 (comment) ? Why didn't you laught at him the same like at me? Oh, because he is your firend, so you can't laughs at your friends, just like you did at me

I'm sorry you have taken it this way, it is not in any evil ways, and trust me we are laughing of each other "Error 40" we just happens to mostly do this on Keybase :smirk:. Next to this I can promise you that @funilrys never laughs (evilly) of everyone, it is always with the best intention from a good haert.

From re-reading https://github.com/funilrys/PyFunceble/issues/13#issuecomment-802307597 I can ensure it is a happy laugh for something went the right way, and you promised to do more tests.

keczuppp commented 3 years ago

Hi,

spirillen : What are you missing in the docs?

Rather nothing, I think now, I didn't get familiar enought with the PyFunceble, thus I just overlooked.

OFF-TOPIC

> spirillen : I'm sorry you have taken it this way, it is not in any evil ways, and trust me we are laughing of each other "Error 40" we just happens to mostly do this on Keybase . Next to this I can promise you that @funilrys never laughs (evilly) of everyone, it is always with the best intention from a good haert. This time it was not so, I felt uncomfortable, I was trying to take it the way you described but I could not. > spirillen : From re-reading #13 (comment) I can ensure it is a happy laugh for something went the right way, and you promised to do more tests. Nope, the fact that I wrote back with complaints later does not change anything, I was just trying to ignore it and think about it like you described, but even 2 days later I still felt uncomfortable to the point I had to send it back to the sender, can't ignore my feelings. In the end, the recipient decides whether he felt offended or not, maybe you should be careful where you put emotes. Best regards.

spirillen commented 3 years ago

...maybe you should be careful where you put emotes.

Best regards.

Well, from my own point of view, I can only say: I'm feeling there are missing some emojis for the "fast feedback", meaning some would have to overlap others.

funilrys commented 3 years ago

but in the embeded Adblock Decoder PyFunceble 4.0.0b39 it's fixed at 66% because site21.com is still not being extracted

I will look at it when I find time to touch that module back.

Yes, it was about "simple way" + the emoji, your comment looks like you wanted to show how stupid I am just because I missed something which you describe as "simple" (a parameter + the fact it should be put in the conjunction with other parameters) in the (big) documentation, which might be not so obvious.

Emojis are not necessarily meant to harm. My emoji was really not meant to harm, nor was it in an evil matter. The usage of the "The simple way" was just meant to introduce something which is not supposed to be hard and something that is supposed to also be easy to use and work as easy as possible. I do not expect everyone to know everything. Even I, have to look into my own source code to find something or provide a better answer to a question. It was not meant to "humiliate", "shame" or "offend" you. If you understood it that way. I'm sorry it was not my intention.

You have to understand that most of the time, the solutions to the problem around here are extremely hard and need a bit of thinking or hacking of me. So I'm extremely happy of myself to present, find or have a simple solution to a problem. I was just happy to have a simple solution. I'm sorry that it was misunderstood.

Was it more funny to you, than your friend being unable to view history of my comment https://github.com/funilrys/PyFunceble/issues/13#issuecomment-797071985 ? Why didn't you laught at him the same like at me? Oh, because he is your firend, so you can't laught at your friends, just like you did at me.

I don't laugh at others. Not that (discrete) way. I'm not like that and it's not my intention. And even when it is the case, it's not in a harmful way.

About the nonreaction to other answers: My time is limited, I don't have time to answer everything. In fact, at the time I'm writing this, I still have around 200 GitHub notifications (Email excluded) to read, answer, sort, and/or put into my Open Source backlog. So most of the time, I'm trying to focus on information that is bringing me more information about what needs to be done or what is actually asked (technically). Side discussions into a feed that are not relevant (at a time X) to what is actually my goal or the goal of the issue are not always my priority. In fact, you may have seen me in the past jumping around multiple comments in the past because what is said is becoming relevant after implementation or a few weeks of changes.

I could laught (by putting a laugh emoji) at him just like you at me, furthermore, I could lught at you (by putting a laugh emoji), every time the Adblock Decoder or PyFunceble crashes making you looking like a fool (but your are a good developer and bugs are normal thing in programming, unavoidable by a human being).

Don't take it too personally, but I'm doing this in my free time. So a good laugh after a long day is not always bad. And sometimes a laughing emoji can be the beginning of a good joke or friendship.

About crashes, PyFunceble 4.0.0 is actually in beta for a good reason. And you are invited to submit all the fatal errors that PyFunceble produces. PyFunceble is not only doing one thing and most of the time it depends on the inputted dataset. As I can't test all possible imaginable datasets - especially when writing a decoder. I'm somehow "bound" to the error report. That's what you indirectly did. And that's what led to a significant change in the source code.

In fact, there was so much feedback on the 4.0.0 version that is the most tested version ever of PyFunceble. Making it probably. one of the less error-prone versions. Nothing is perfect but there is hope that this new version brings less error and more stability.

Really? Then why didn't you explain what was the purpose of the emoiji in your comment then.

Why should someone lose time to explain all single nontechnical decisions when it's not necessary? I didn't judge it necessary. Now I still took the time to explain...

But the fact is: I do have private, professional, familial, and public (through here) lives. So each minute I lose-d to explain an emoji or the choice of an X or Y word in a sentence is a time I could use to do something else to actually answer more technical questions or simply help move forward.

An emoji shouldn't be given that much time and energy. It happened, I understand it offended you or made you uncomfortable. I can't promise that you won't be offended again as I can't speak for others who use this platform. But be sure that your message and feelings came "laud and clear".

Do you want to just tell me you put the emoji for no reason, or just because you were in a happy mood and just by accident it was looking like you were laughting at me...

I wasn't laughing at you nor at the situation. If I would, I would have used:

Or is it not appropriate enough?

I don't believe your cheap explanation, lie to yourself, I spent much time analysing, whether your intention was to laught at me or not, and something said me very clear you were.

Believe it or not, when I say such a thing, I mean it. My intention was and is not to laugh at you.

abused by trolls, they abuse emojis to troll other people at every occasion

Why would I literally troll others on "every occasion" (or not) on GitHub when I have other things to do. I just want to move forward, help, and code if I get the chance and the time for it. It's all in my free time. Troll do "their thing" all day long, probably in their free time too but I believe that they are not as busy as some of us.

I consider you a positive person and great developer overall, but just don't do it again.

Thank you for the compliment. I'm indeed a positive person. I'm not looking to harm anyone. I was harmed enough in my life to know. I didn't think that that sentence with an emoji will harm someone. People who know me, know that I'm not a troll or someone who constantly "humiliate", "shame" and/or "offend" someone because of a lack of knowledge. I know that not everyone has the same knowledge. That's why I'm always happy to provide some of my expertise, knowledge, and help across multiple projects.

By the way, the word "fvcktard" was not necessary. I'm polite enough, and even if an emoji offended you, it's not a reason to use such a word. That's something that offended me but I chose to ignore it at the time I read it. Please avoid such language in the future.

Sorry for the misunderstanding. Cheers.

keczuppp commented 3 years ago

OFF-TOPIC

Hi funilrys, as for the incident: I appreciate your explanations, but I will not change my mind, my answer is the same as to spirillen: https://github.com/funilrys/PyFunceble/issues/13#issuecomment-805941359, there is too many people who tell sweet lies and manipulate, I trust my feelings. (even if you did it unintentionally, still you did it, and the other had right ot get pissed, but I still think you did it intentionally.) as for: > funilrys : https://github.com/funilrys/PyFunceble/issues/13#issuecomment-806238322 : about crashes (...) Sure, and that's why I mentioned the crashes **only as an example** where someone could dishonestly and groundlessly laugh at someone for no real reason, so why all the explanations...you see the point...: even despite that it was just an example, you write back as if you took it seriously, and what if I would write it seriously and laught emote it (because for. ex. I would be happy because I spot a bug) I bet it wouldn't be pleasant for you. :smiley: as for: > funilrys : https://github.com/funilrys/PyFunceble/issues/13#issuecomment-806238322 : > > keczuppp : Really? Then why didn't you explain what was the purpose of the emoiji in your comment then. > > Why should someone lose time to explain all single nontechnical decisions when it's not necessary? I didn't judge it necessary. Now I still took the time to explain... Nah, either you got it wrong, or it was me who worded it wrongly, the logic of my question was different: it was not about "`explain purpose of the emoji` `in the comment`" - it was not about to `explain in the comment`, it was about to: `explain purpose of` `the emoji in the comment` - it was about to explain `emoji in the comment`, (`emoji which is in your comment`), the part `in the comment` reffers to the actual placement of the emoji, not where to put the explanations. I think I would use a comma `,` if I meant to put the explanations in the comment, an example: `Really? Then why didn't you explain what was the purpose of the emoiji, in your comment then.` or perhaps I should use rather **from** than **in**: "of the emoji ~~**in**~~ **from** you comment". The whole point of the question was that you begun to explain in your comment: https://github.com/funilrys/PyFunceble/issues/13#issuecomment-803145752 what **was not** purpose of the emoji, but at the same time, you did not explain what **was** the purpose. I have no idea whose fault it is for this misunderstanding in grammar. > funilrys : https://github.com/funilrys/PyFunceble/issues/13#issuecomment-806238322 : > > keczuppp : abused by trolls, they abuse emojis to troll other people at every occasion > > Why would I literally troll others on "every occasion" (or not) on GitHub when I have other things to do. I just want to move forward, help, and code if I get the chance and the time for it. It's all in my free time. Troll do "their thing" all day long, probably in their free time too but I believe that they are not as busy as some of us. I didn't mean to say you are a troll, but the division of people is not limited to the black / white case you provided: to those who always troll and to those who never troll, but also there are those who troll occasionally only. > By the way, the word "fvcktard" was not necessary. I'm polite enough, and even if an emoji offended you, it's not a reason to use such a word. That's something that offended me but I chose to ignore it at the time I read it. Please avoid such language in the future. I am not a person who avoids defense against harassment, I prefer to defend myself and choose the methods of defense myself. If you feel offended, it's because I felt offended by you first and decided to pay back. Even if you did it unintentionally, still you did it, and the other has right to get pissed, be careful where you paste your pokemons, no one is clairvoyant to guess the intentions of other people's emotes, before you put an emoji, think twice whether it might look like laughting at someone or not, if you still want to put an emoji but at the same time you know it might look like laughting at someone, put some short explanations of purpose of the emoji, do not put your interests ahead of the interests of others, just because you are happy and have a need to put an emoji, doesn't mean you can put it anywhere, anytime, in a way it might offend others. Please be carefull with emojis at me in the future. Also it would be better if you guys created a separate issue for the off-topic, you made the Adblock Decoder thread a garbage can. After I felt offended, I expressed it in a single short sentence: https://github.com/funilrys/PyFunceble/issues/13#issuecomment-802576296, if you don't agree with it and wanted to continue discussion and turn it into a longer off-topic thread, you could create a separate issue for the whole incident. Now, is the off-topic over finally? Can we get back to the Adblock Decoder please? I would rather spend time on something more constructive / technical than long off-topic conversations which are very time consuming and can end in an unpleasant escalation. Why this whole show. Guys come on, lets end this show finally, no need to turn it into a drama festival. Best regards.

spirillen commented 3 years ago

Please, can we park that emoji?? agreeing that you do not agreeing in the explanation and usage of it for the the specific situation??

And let's stick to the Code of conduct and get back to the actual topic, the error produced.

I'll hope so.

keczuppp commented 3 years ago

funilrys, can we get some cleaning in this thread, could you put in the spoiler your OFF-TOPIC https://github.com/funilrys/PyFunceble/issues/13#issuecomment-806238322, just like I did with my OFF-TOPICS, thx

keczuppp commented 3 years ago

OK, so I've just tested the newest PyFunceble dev right now and I've noticed that the reported issues mentioned in : https://github.com/funilrys/PyFunceble/issues/13#issuecomment-749607193 and https://github.com/funilrys/PyFunceble/issues/13#issuecomment-803537932 have been fixed.

The summarision:

keczuppp: As for the last 3 failures, many of such failures can be found in https://easylist-downloads.adblockplus.org/easylistpolish.txt The list contains about 2961 domains, but only 2459 are found by Adblock Decoder (with --aggressive option), which gives 83% efficiency.

the current EasyList Polish contains 3190 domains and the newest Pyfunceble has extracted 3106 what gives 97% efficiency compared to 83% previously

keczuppp: currently the decoder extracts about a half of domains, to prove it I copy-pasted all domain= lines from several popular / big adblock filter lists + two polish lists: EasyList, EasyPrivacy, AdGuard Base, AdGuard Tracking Protection, Official Polish Filters for AdBlock, uBlock Origin & AdGuard, EasyList Polish and put into a single file / list (see domain=.zip), which contain about 11425 domains, but Adblock Decoder extracts only about a half (5426 domains)

the newest PyFunceble has extracted 8298 what gives 73% compared to previously 48%

Good improvement.

funilrys / PyFunceble

Adblock decoder ignore some portion when decoding #13