funilrys / PyFunceble

The tool to check the availability or syntax of domain, IP or URL.
https://pyfunceble.github.io
Apache License 2.0
297 stars 44 forks source link

BUG: dead domain query #343

Open ryanbr opened 1 year ago

ryanbr commented 1 year ago

More of a general question:

qbqzbzbzzzjjb.top                                                                                    INACTIVE    STDLOOKUP
qbrrrlawwrvbv.top                                                                                    INACTIVE    STDLOOKUP
qmqjvreqwywbk.top                                                                                INACTIVE    STDLOOKUP
qmykwjryyrrjj.top                                                                                    INACTIVE    STDLOOKUP

Are these considered INACTIVE domains?

These are recent addions. and are being marked as INACT

funilrys commented 1 year ago

Hi @ryanbr,

sorry for the waiting time. It was my WeAreDevs Congress Week. Anyway, it seems that those domains are unreachable from the perspective of PyFunceble.

However, I can replicate you result but only if I disable the WHOIS lookup. Indeed:

Therefore, PyFunceble flagged them as INACTIVE.

Hope that helps. Let me know if you need something else.

Best Regards, Nissar

ryanbr commented 1 year ago

Which command line flags do I need to ensure they're marked correctly?

docker run -it pyfunceble/pyfunceble --color --aggressive --adblock -f https://secure.fanboy.co.nz/easylist.txt > el.txt

What I'm currently using

funilrys commented 1 year ago

Hey @ryanbr, that's strange ... Here it's working:

cat test.list ; docker run -it -v ./test.list:/tmp/test.list pyfunceble/pyfunceble  -f /tmp/test.list
qbqzbzbzzzjjb.top
qbrrrlawwrvbv.top
qmqjvreqwywbk.top
qmykwjryyrrjj.top
    ██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
    ██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
    ██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
    ██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
    ██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
    ╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Subject                                                                                              Status      Source    
---------------------------------------------------------------------------------------------------- ----------- ----------
qbrrrlawwrvbv.top                                                                                    ACTIVE      WHOIS     
qbqzbzbzzzjjb.top                                                                                    ACTIVE      WHOIS     
qmqjvreqwywbk.top                                                                                    ACTIVE      WHOIS     
qmykwjryyrrjj.top                                                                                    ACTIVE      WHOIS     

Status      Percentage   Amount      
----------- ------------ ------------
ACTIVE      100%         4           
INACTIVE    0%           0           
INVALID     0%           0           

                          █
                        ████    ██
                     ███ ██   █████   ███
               ███ █████    █████   █████
             ██████████   █████   █████
           ██████████   █████   ██████   ██
         ██████████   █████   ██████   ████
       ███████████████████████████   █████
      ██████████████████████████   ████
     ████████ █████ ██████████   █████
     ███████   ███   ███████   █████
     █████████████████████   █████
     ████████████████████  █████
     ███████         █████████
      ████████     █████████
        ██████████████████
          ██████████████
               ████

Thank you for using PyFunceble!

I will let the test run a few 100 times to see if the result is a different one. I'll let you know

funilrys commented 1 year ago

Okay, so there is apparently a rate limit because PyFunceble is querying too quickly 😞 It's maybe time to launch a collaborative platform that will look into those whois record at a very slow speed.

funilrys commented 1 year ago

@ryanbr it's not a solution, but a workaround is to use the --cooldown feature to make PyFunceble sleep a few milliseconds / seconds between each test... It's suboptimal, but it seems to work ...

ryanbr commented 1 year ago

Using a --coldown 5, unless this is too short?

jvnvrqzvyvbmv.top                                                                                    INACTIVE    STDLOOKUP 
jvaemllmeezyw.top                                                                                    INACTIVE    STDLOOKUP 
jvylyreyazmea.top                                                                                    INACTIVE    STDLOOKUP 
kjvaqbyebqkzy.top                                                                                    INACTIVE    STDLOOKUP 
laqwnrazvywaz.top                                                                                    INACTIVE    STDLOOKUP 
lazmblmywqjyb.top                                                                                    INACTIVE    STDLOOKUP 
lrqvmaverkabb.top                                                                                    INACTIVE    STDLOOKUP 
lrvrayrnqjzkz.top                                                                                    INACTIVE    STDLOOKUP 
lzylbjlqeknwe.top                                                                                    INACTIVE    STDLOOKUP 
ryanbr commented 1 year ago

Another test: --cooldown 55 (really slow, overnight..) Maybe a better .top whois registerer is needed?

qbqzbzbzzzjjb.top                                                                                    INACTIVE    STDLOOKUP
qbrrrlawwrvbv.top                                                                                    INACTIVE    STDLOOKUP
qmqjvreqwywbk.top                                                                                    INACTIVE    STDLOOKUP
qmrwbzwalrmry.top                                                                                    INACTIVE    STDLOOKUP
qmvnrvjeyyzna.top                                                                                    INACTIVE    STDLOOKUP
qmykwjryyrrjj.top                                                                                    INACTIVE    STDLOOKUP
qqvvkebebnbb.top                                                                                     INACTIVE    STDLOOKUP
qqyqeqqmqrybb.top                                                                                    INACTIVE    STDLOOKUP
qwmaeqrneewny.top                                                                                    INACTIVE    STDLOOKUP
funilrys commented 1 year ago

I didn't expect that 🤔. We definitely need a better .top whois register - then.

ryanbr commented 1 year ago
qiqgvcrnhwc.com                                                                                      INACTIVE    STDLOOKUP
ncpxhrurirscgsd.com                                                                                  INACTIVE    STDLOOKUP 
ndryqdqwtnn.com                                                                                      INACTIVE    STDLOOKUP 
nceqpmxyhqo.com                                                                                      INACTIVE    STDLOOKUP
zheevmmvlwg.com                                                                                      INACTIVE    STDLOOKUP

Doing a whois is showing some recent domain being marked as INACTIVE. --cooldown 7 was used here.