BUG: urls in domain lists....

spirillen commented 1 year ago

Description

After testing same file, but first the list contained proto://domain and I initiated the test with the -f. Once I spotted the failure, I stopped the process and changed the test parameter to -uf, but as I actually only had domains in the list, I once again stopped the test, removed the protocol from the lists and restarted the test process with -f

The result... (pyfunceble/output/pr-list.txt/domains/ACTIVE/list)

# Generated by PyFunceble (v4.2.0a8.dev) / https://git.io/vpZoI
# Date of generation: 2023-05-01T15:08:46.655285

https://23490.vochtblog.be
23490.vochtblog.be
313.biz
https://566.biz
566.biz
7175.biz
715517.biz
https://20200813085002.maygsimpex.biz
20200813085002.maygsimpex.biz
75733.prohoster.biz
https://527mxxxx.code.blog
527mxxxx.code.blog
https://999zxxxilb.music.blog
999zxxxilb.music.blog
https://77076.comvocenaweb.com.br
77076.comvocenaweb.com.br
https://98g.hpg.com.br
98g.hpg.com.br

Configuration

cli_testing:
autocontinue: true
cidr_expand: true
db_type: mariadb
display_mode:
all: true
colour: true
dots: false
execution_time: true
less: false
percentage: true
quiet: false
simple: false
status: all
file_generation:
analytic: true
hosts: false
merge_output_dirs: false
no_file: false
plain: true
unified_results: false
local_network: false
sorting_mode:
hierarchical: true
debug:
active: true
level: critical
lookup:
collection: false
dns: true
http_status_code: true
netinfo: true
reputation: false
special: true
timeout: 5
whois: true
user_agent:
browser: firefox
custom: null
platform: linux
dns:
follow_server_order: false
trust_server: true
protocol: UDP
server:
- 192.0.2.2
- 127.0.0.1:port
collection:
preferred_status_origin: recommended
push: true
url_base: https://collection.dead-hosts.funilrys.com
proxy:
global:
http: socks5h://127.0.0.1:9050
https: socks5h://127.0.0.1:9050
rules:
- http: socks5h://127.0.0.1:9050
https: socks5h://127.0.0.1:9050
tld:
- onion

Reproduction

See above

Expected behavior

Screenshots

Versions

OS: Ubuntu 22.04.1

Python Version: 3.10

PyFunceble Version: 4.2.0a8.dev

Additional context

funilrys commented 1 year ago

That it interesting. I never through of such a use case when I wrote the decoder years ago.

Let's find a way to aggressively decode hosts file too.

spirillen commented 1 year ago

Yep, that is a weird one :smile: not to replicate for anyone

funilrys / PyFunceble