The last domain is missed sometimes

[keczuppp]

Bug description

The last domain from the file is ignored / missed / skipped sometimes.

Modifications under .PyFunceble.yaml

A clear representation of what you changed.

Reproduction

Steps to reproduce the behavior:

Content of the file test.txt:

webpage1.com
webpage2.com
webpage3.com

1. Run command pyfunceble -f test.txt several times.
2. Notice sometimes the last domain is skipped.

Expected behavior

The last domain should not be skipped.

Screenshots

Versions

OS: Win 7 SP1 Python Version: 3.7.6 PyFunceble Version: PyFunceble-dev 4.0.0b41

Additional context

Add any other context about the problem here.

The log:

D:\download_big_temp\_koding>pyfunceble --syntax -f test.txt

########  ##    ## ######## ##     ## ##    ##  ######  ######## ########  ##       ########
##     ##  ##  ##  ##       ##     ## ###   ## ##    ## ##       ##     ## ##       ##
##     ##   ####   ##       ##     ## ####  ## ##       ##       ##     ## ##       ##
########     ##    ######   ##     ## ## ## ## ##       ######   ########  ##       ######
##           ##    ##       ##     ## ##  #### ##       ##       ##     ## ##       ##
##           ##    ##       ##     ## ##   ### ##    ## ##       ##     ## ##       ##
##           ##    ##        #######  ##    ##  ######  ######## ########  ######## ########

You are using the Beta version of PyFunceble 4.0.0!
Please take the time to communicate with us when you notice
something unusual.

Subject
 Status      Source
----------------------------------------------------------------------------------------------------
 ----------- ----------
←[30m←[42mwebpage1.com
           VALID       SYNTAX
←[30m←[42mwebpage2.com
           VALID       SYNTAX

Status      Percentage   Amount
----------- ------------ ------------
VALID       100%         2
INVALID     0%           0

Thank you for using PyFunceble!

D:\download_big_temp\_koding>pyfunceble --syntax -f test.txt

########  ##    ## ######## ##     ## ##    ##  ######  ######## ########  ##       ########
##     ##  ##  ##  ##       ##     ## ###   ## ##    ## ##       ##     ## ##       ##
##     ##   ####   ##       ##     ## ####  ## ##       ##       ##     ## ##       ##
########     ##    ######   ##     ## ## ## ## ##       ######   ########  ##       ######
##           ##    ##       ##     ## ##  #### ##       ##       ##     ## ##       ##
##           ##    ##       ##     ## ##   ### ##    ## ##       ##     ## ##       ##
##           ##    ##        #######  ##    ##  ######  ######## ########  ######## ########

You are using the Beta version of PyFunceble 4.0.0!
Please take the time to communicate with us when you notice
something unusual.

Subject
 Status      Source
----------------------------------------------------------------------------------------------------
 ----------- ----------
←[30m←[42mwebpage1.com
           VALID       SYNTAX
←[30m←[42mwebpage2.com
           VALID       SYNTAX
←[30m←[42mwebpage3.com
           VALID       SYNTAX

Status      Percentage   Amount
----------- ------------ ------------
VALID       100%         3
INVALID     0%           0

Thank you for using PyFunceble!
Share your experience on Twitter with #PyFunceble or @PyFunceble!
Do you have a feedback, an issue or an improvement idea? Let us know on GitHub!

[spirillen]

I believe this can have relation to the worker as in #222 & #229, also I have reported a similar issue in another issue :thinking: Can't remember witch :thought_balloon:

[funilrys]

Uhh, this will take more time that I thought.

I tested 60 times, without any error.

$ export DEBUG_PYFUNCEBLE=yes
$ cat test.list
webpage1.com
webpage2.com
webpage3.com
$ for i in {1..60}; do echo "$i $(pyfunceble --syntax -f test.list --logging-level debug  | grep -E "%" | awk '{ print $3 }' | head -1)"; done
1 3
2 3
3 3
4 3
5 3
6 3
7 3
8 3
9 3
10 3
11 3
12 3
13 3
14 3
15 3
16 3
17 3
18 3
19 3
20 3
21 3
22 3
23 3
24 3
25 3
26 3
27 3
28 3
29 3
30 3
31 3
32 3
33 3
34 3
35 3
36 3
37 3
38 3
39 3
40 3
41 3
42 3
43 3
44 3
45 3
46 3
47 3
48 3
49 3
50 3
51 3
52 3
53 3
54 3
55 3
56 3
57 3
58 3
59 3
60 3

[keczuppp]

funilrys : I tested 60 times, without any error.

(4.0.0b39)

Because you tested with altered/different command than provided by me in the first comment, neither me can reproduce with --logging-level debug, however as soon I remove --logging-level debug I can reproduce. but yeah, strange that adding --logging-level debug makes a difference in reproducibility of the bug (did I just spot another bug or is the original bug so nasty...)

=========================================================================== ( 4.0.0.b40)

The above was still on 4.0.0b39 I've retested on 4.0.0.b40 right now and it's even worse - it always skips the last domain both with and without --logging-level debug, and it does it everytime (not only at the first run).

=========================================================================== ( 4.0.0.b41)

Now behaviour with --logging-level debug is still the same like in 4.0.0.b40. But behaviour without --logging-level debug is (4 runs): 2 3 2 2 - which means only in the 2nd run it found all the 3 domains.

As seen, each new version gives different behaviour (strange), but still not fixed.

[funilrys]

Hi @keczuppp,

I tested 60 times again without the --logging-level argument and I get the same result anyway.

I'll try to look at the root of this today.

Have a nice day/night.

[keczuppp]

I've retested on the newest version, see my updated comment: https://github.com/funilrys/PyFunceble/issues/230#issuecomment-808416716 Btw I didn't check whether anything related to the issue was changed in 40 or 41, so I don't know whether the different behaviour of the issue at my side is related to changes in 40 or 41 or is it just a coincidence (which would mean it is the bug crazy itself).

[spirillen]

Python Version: 3.7.6

Python <= 3.7 is not supported...., it stroke me, that the similar problem we had before Christmas was on my Pi4 with python 3.7.x

https://pyfunceble.readthedocs.io/en/dev/facts/known_issues.html#python-3-7

Could you try to update your python to 3.9.x and test again?? for example in a Conda environment?

[keczuppp]

OS: Win 7 SP1

I can't upgrade because 3.7.6 is the last version whch runs onWin 7 SP1, I already tried in the past to install the newest Python versions but anything newer than 3.7.6 throws an error saying it requires minimum Win 8.1 or 2012.

OFF-TOPIC

- not that I couldn't install Win10 on my PC, but there are 2 reasons I won't: - Win10 is a little ugly, Win7 has a beautiful Aero - Win10 spies much, unlike Win7 SP1 (Win7 versions above SP1 do spy as well) - nor I have a disk space to install Win10 on Virtual Machine nor I like to spend time and effort to do it. -

However, I've checked on another device, a laptop which has Win10, but the bug is still here with Python 3.9.2 as well. (I didn't try in Conda environment)

[spirillen]

Why not just upgrade to *nix?? I did for the same reasons you mentioned. And trust me, the moon have to fall down or burn up or something before I'm changing back to shitdows or IOS.x-predetorian-animal

[funilrys]

I tried to get a Windows 10 with Python 3.9.2 and 3.7.6 and I could not reproduce this issue (again).

Can you run with --logging-debug and send us a ZIP of the logging directory ?

[keczuppp]

spirillen : Why not just upgrade to *nix?

Shortly, tried in the past, but had bad experiences, more or less like similiar to such many topics: https://forum.beyond3d.com/threads/linux-is-a-pile-of-shit.60616/. More in the spoiler:

OFF-TOPIC (to spirillen)

Tried various Linux distros in the past, but 3 reasons caused me to drop Linux: - it's too problematic, there are too many bugs / problems, when you fix one bug / problem, three more pop up and so on ... you spend half your time on google searching for how to fix Linux bugs that never end ... I'm not saying that on Windows there are no problems, but far fewer than on Linux - while many Windows applications have their alternatives in Linux or can run in Wine, still there is a lack of some good applications, that are present under Windows, and they don't run in Wine at all, or have problems with it - also these who says Linux is safer than Windows, must be joking...it might look like safer, but it's just an illusion, it's "safe" only because it has only 1-2% market share compared to Windows 76%, thus not many people are interested to hack Linux nor there is many people to discover all the security holes, once Linux would reach same market share like Windows, it would become a target to hack and would become as leaky and dangerous as Windows and vice-versa as well, when Windows 's market share would drop to 1-2%, Windows would become "safe" like Linux is now, because not many would be interested to hack it or to find security holes... And that's why Linux has only 1-2% market share as a desktop OS... I would say that Linux is a crap as a desktop OS. A good thing as for a desktop OS, is Unity GUI, a good equivalent of Aero GUI, unfortunately Unity has been dropped in Ubuntu. However Linux is great for servers, routers, firewalls, supercomputers and some other usage scenarios. So the two systems are great, just they have their own separate different usage scenarios.

==========================================================================

funilrys : Can you run with --logging-debug and send us a ZIP of the logging directory ?

Done (on Win7 SP1 machine), I've zipped whole output folders as a proof and in case you need it in the future, see the attachements:

output (a bad run - only 2 domains).zip

output (a good run - 3 domains).zip

A NOTE: it seems a behaviour of the bug is random, sometimes it prints 3 domains, sometimes just 2, no rules, also sometimes 3 domains several times in a row, sometimes 2 domains several times in a row. Just like in a casino, lol.

[ZeroDot1]

I tried to reproduce the bug, but I did not succeed. With the current AUR version the test works without problems. Testfile: test.txt

It is very important that you always use the latest versions, then there are usually fewer errors. Furthermore I recommend to use ArchLinux.

[keczuppp]

I've noticed that the bug exists even without --syntax:

LOG

``` D:\download_big_temp\_koding>pyfunceble -f test.txt ######## ## ## ######## ## ## ## ## ###### ######## ######## ## ######## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## #### ## ## ## #### ## ## ## ## ## ## ## ######## ## ###### ## ## ## ## ## ## ###### ######## ## ###### ## ## ## ## ## ## #### ## ## ## ## ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## ## ## ## ####### ## ## ###### ######## ######## ######## ######## You are using the Beta version of PyFunceble 4.0.0! Please take the time to communicate with us when you notice something unusual. Subject Status Source ---------------------------------------------------------------------------------------------------- ----------- ---------- ←[30m←[42mwebpage2.com ACTIVE WHOIS ←[30m←[42mwebpage1.com ACTIVE WHOIS Status Percentage Amount ----------- ------------ ------------ ACTIVE 100% 2 INACTIVE 0% 0 INVALID 0% 0 Thank you for using PyFunceble! Share your experience on Twitter with #PyFunceble or @PyFunceble! ```

[ZeroDot1]

I've noticed that the bug exists even without --syntax:

I can't confirm this, everything runs smoothly for me.

[keczuppp]

https://github.com/funilrys/PyFunceble/commit/bda224ea25b2595f96bf7ed2b5874837472c2456 : This patch (possibly) fixes|touches #230

40.0.0.b42 The patch doesn't fix the bug, but it makes the bug occurring every time, what means it does touch it, what is a good news, we know where the bug lies, so now it can be fixed finally.

[ZeroDot1]

The patch doesn't fix the bug,

I can't confirm this, everything runs smoothly through like liquid butter.

@funilrys Very good work, thank you very much. i appreciate your work very much. :1st_place_medal:

[keczuppp]

keczuppp : https://github.com/funilrys/PyFunceble/issues/230#issuecomment-812604774

In 4.0.0.b44 behaviour of the bug is back to random.

[spirillen]

@keczuppp can you enable the debug log as @funilrys suggested previously and then share it with us?

Documention on subject: https://pyfunceble.readthedocs.io/en/dev/usage/index.html#logging-level

[keczuppp]

• done: output_bug.zip + output_no_bug.zip

• I run: D:\download_big_temp\_koding>pyfunceble --logging-level debug --syntax -f test.txt 2 times

• the first run resulted no bug occured (all domains present)

• the second run resulted the bug occured (the last domain was missing)

• before every of the both runs, the output folder was removed

[spirillen]

Super, Ping @funilrys

[funilrys]

Ping

[spirillen]

:ping_pong: Pong :smile:

@keczuppp Is it solved by https://github.com/funilrys/PyFunceble/commit/38beaee2c5a714ef2159a95c1de825d50b4d663d (v4.0.0b47) ?

Another thing from your logs... Are you running on Win 7 32bit Single core/Thread? as it appears you are only using one core in the tests.

[keczuppp]

It is finally solved by v4.0.0b47. I was using Win7 SP1 64bit, with default pyfunceble settings.