Closed spirillen closed 3 years ago
Ping @smed79 @ryanbr @Khrin @DandelionSprout @THEtomaso @Yuki2718 Please comment what you'll find of errors or suggestions for tests :smiley:
And Thank you so much for your good work with the EasyList project
UPDATE: Thanks to @keczuppp in https://github.com/funilrys/PyFunceble/issues/156#issuecomment-751723455 my laziness caught up with me, sorry
First report
The --adblock
argument do not accept more than one argument vs the -f
argument where you can assign several sources at once
Reported at: https://github.com/funilrys/PyFunceble/issues/156
https://travis-ci.com/github/dns-test/PyFunceble-Sandboxed/builds/210574987
I have been using this option combined with -f
(-ad -f
), at least it seems to recognize not only domain rules (||example.com^
) but also cosmetic and scriptlet (##
), but haven't tested thoroughly. Only annoyance I get is it complains about cp932 codec when the file includes non-ascii characters so I have to remove them from the file. But it seems to be more of a Python issue and also specific to my Windows (Japanese locale).
Only annoyance I get is it complains about cp932 codec when the file includes non-ascii characters so I have to remove them from the file. But it seems to be more of a Python issue and also specific to my Windows (Japanese locale).
What file are you using? is it a public one I can test with?
PS: the -ad
is replaced with --adblock
in version 4
What file are you using? is it a public one I can test with?
For example, foreign.txt in AdGuard Base filter (formerly English filter). This is the actual error message:
Super, I setup a test to have that fixed (Travis)
@funilrys , maybe you could spare a moment to setup a GH-Action to make windows + cygwin tests in the sandbox, you seems to have been working a bit with this
Have you tried https://stackoverflow.com/a/52617143/5308363 ? I had a simular problem with the GitHub Actions under Windows and I solved it by adding the PYTHONIOENCODING
variable: https://github.com/funilrys/PyFunceble/blob/d97612786227c0d7059c733ba4e61fcba51b29ba/tox_run.ini#L8
My experiences with --adblock
in these past 2 days:
I often run into ConnectionAbortedError: [Errno 113] Software caused connection abort
, and sometimes even Fatal Error: [Errno 113] Software caused connection abort
, most often when running PyFunceble on Nordic Filters (by saving the main list version locally and running PyFunceble --adblock -f NorwegianList.txt
).
Comparatively, doing the same with EasyList Lithunia, which admittedly targets the ABP syntax instead of the uBO syntax, does not seem to cause this type of errors at all.
Have you tried https://stackoverflow.com/a/52617143/5308363 ?
So run chcp 65001 & set PYTHONIOENCODING=utf-8
before Pyfunceble? I'll check later.
I often run into
ConnectionAbortedError: [Errno 113] Software caused connection abort
, and sometimes evenFatal Error: [Errno 113] Software caused connection abort
, most often when running PyFunceble on Nordic Filters (by saving the main list version locally and runningPyFunceble --adblock -f NorwegianList.txt
).
Haven't got that on any of uAssets, AG, or personal lists.
@DandelionSprout what is your test string?
Exmaple of what I mean:
pyfunceble --share-logs -dbr 6 -ex --dns 192.168.1.104:53 -w 6 -a --database-type mariadb --no-files --local -f somefile
pyfunceble --adblock -f https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/EnglishFilter/sections/foreign.txt \
--dns 192.168.1.104 192.168.1.103 192.168.1.102 192.168.1.101 \
127.0.0.1 192.158.1.2 192.158.1.3 192.158.1.4 192.158.1.5
My test result is: SLOW, but my eyes haven't court any weird
@Yuki2718 could you retest this issue with @funilrys suggestion?
Another thing I noticed, we aren't filtering what is to become doubles...
mobilarena.hu ACTIVE DNSLOOKUP
mobilarena.hu ACTIVE DNSLOOKUP
mobilarena.hu ACTIVE DNSLOOKUP
filmvilag.org ACTIVE WHOIS
filmvilag.org ACTIVE WHOIS
filmvilag.org ACTIVE WHOIS
gyorietokc.hu ACTIVE DNSLOOKUP
gyorietokc.hu ACTIVE DNSLOOKUP
gyorietokc.hu ACTIVE DNSLOOKUP
bien.hu ACTIVE DNSLOOKUP
bien.hu ACTIVE DNSLOOKUP
itcafe.hu ACTIVE DNSLOOKUP
itcafe.hu ACTIVE DNSLOOKUP
hardverapro.hu ACTIVE DNSLOOKUP
hardverapro.hu ACTIVE DNSLOOKUP
hardverapro.hu ACTIVE DNSLOOKUP
hardverapro.hu ACTIVE DNSLOOKUP
139.99.209.98 ACTIVE DNSLOOKUP
139.99.209.98 ACTIVE DNSLOOKUP
139.99.209.98 ACTIVE DNSLOOKUP
139.99.209.98 ACTIVE DNSLOOKUP
139.99.209.98 ACTIVE DNSLOOKUP
139.99.209.98 ACTIVE DNSLOOKUP
Test of @DandelionSprout https://github.com/funilrys/PyFunceble/issues/155#issuecomment-751532410
pyfunceble --adblock --dns 192.168.1.104 192.168.1.103 192.168.1.102 192.168.1.101 \
127.0.0.1 192.158.1.2 192.158.1.3 192.158.1.4 192.158.1.5 \
-f https://raw.githubusercontent.com/DandelionSprout/adfilt/master/NorwegianList.txt \
https://raw.githubusercontent.com/EasyList-Lithuania/easylist_lithuania/master/easylistlithuania.txt
Result: No errors
@spirillen Sure, I just forgotten. As I said, I believe the issue is specific to Japanese locale.
Doesn't help, I still get the same error.
My experiences with
--adblock
in these past 2 days:I often run into
ConnectionAbortedError: [Errno 113] Software caused connection abort
, and sometimes evenFatal Error: [Errno 113] Software caused connection abort
, most often when running PyFunceble on Nordic Filters (by saving the main list version locally and runningPyFunceble --adblock -f NorwegianList.txt
).Output (Very long): Comparatively, doing the same with EasyList Lithunia, which admittedly targets the ABP syntax instead of the uBO syntax, does not seem to cause this type of errors at all.
Do you have a firewall configured? I'm asking because it is one of the ways to get ConnectionAbortedError: [Errno 113] Software caused connection abort
.
Seems like this works without any noticeable problems as of 4.0.0b1.dev. I'm not 100% sure how at the time of writing, but it does.
Hey @Yuki2718 Is this one still an issue https://github.com/funilrys/PyFunceble/issues/155#issuecomment-751323033
If so, I think I would suggest to put it into it's own issue, as I don't see why you shouldn't be allowed to run a full sized scan as well...
@spirillen I will test in a few days, please wait.
Thank you @Yuki2718! Can you test with 4.0.0bX
?
On 4.0.0b19 the issue is solved, and it's noticeably faster than the old version. Thx for keeping good work!
BTW I had to downgrade Python to 3.8 as dnspython is not yet compatible to 3.9.
dnspython
?? I'm running dnspython on python 3.9.1 without any issues
python3.9 -m pip list | grep dnspython
dnspython 2.0.0
dnspython in /usr/local/lib/python3.9/dist-packages (2.0.0)
And a HUGE thanks for your feedbacks @Yuki2718
Until now there have been non test of the
--adblock
variable....This should be the thread for that purpose.
I'll be using a valid list for this purpose from EasyList, more specific I'll be using the following list to get as broad a number of rules as possible and yet keeping things small.
If you find any test case, you think I have forgotten please post a comment.
NOTE
I'll update this particular post with any news rather than comments, to better maintain a overview of what is done and have to be done.
@funilrys Please add to this post to https://github.com/funilrys/PyFunceble/projects/6