Open p5pRT opened 9 years ago
p5pRT
commented
9 years ago threads.pm says:
The use of interpreter-based threads in perl is officially discouraged.
Why is that? Is it just because they're rather heavy? If so\, why not just say so and let the user decide whether he wants to use them or not? After all\, heavy threads aren't a problem if you reuse them (as in a worker model).
p5pRT
commented
9 years ago On 05/05/2015 05:23 PM\, Eric Brine (via RT) wrote:
# New Ticket Created by "Eric Brine" # Please include the string: [perl #125106] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125106 >
threads.pm says:
The use of interpreter-based threads in perl is officially discouraged.
Why is that? Is it just because they're rather heavy? If so\, why not just say so and let the user decide whether he wants to use them or not?
+1
p5pRT
commented
9 years ago The RT System itself - Status changed from 'new' to 'open'
p5pRT
commented
9 years ago On Tue\, 5 May 2015 08:23:15 -0700 "Eric Brine" (via RT) \perlbug\-followup@​perl\.org wrote:
# New Ticket Created by "Eric Brine" # Please include the string: [perl #125106] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125106 >
threads.pm says:
The use of interpreter-based threads in perl is officially discouraged.
Why is that? Is it just because they're rather heavy? If so\, why not just say so and let the user decide whether he wants to use them or not? After all\, heavy threads aren't a problem if you reuse them (as in a worker model).
Or people could help me finish writing/editing the perlconcurrency doc\, so one day it will explain in more detail:
http://www.leonerd.org.uk/code/perlconcurrency.pod
-- Paul "LeoNerd" Evans
leonerd@leonerd.org.uk http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS
p5pRT
commented
9 years ago On May 5\, 2015\, at 11:23\, Eric Brine \perlbug\-followup@​perl\.org wrote:
# New Ticket Created by "Eric Brine" # Please include the string: [perl #125106] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125106 >
threads.pm says:
The use of interpreter-based threads in perl is officially discouraged.
Why is that? Is it just because they're rather heavy? If so\, why not just say so and let the user decide whether he wants to use them or not? After all\, heavy threads aren't a problem if you reuse them (as in a worker model).
If your documentation is anything like mine\, the paragraph before that statement tries to explain why. Perhaps the discouragement should begin with "Because of this\, the use of...." to hint that the reader may have skipped something important and to discourage people from taking a provocative statement out of context.
The discouragement was added on Mar 2\, 2014 after an intense deliberation on perl5-porters:
"RFC: add discouragement warning to perl threads documentation" http://www.gossamer-threads.com/lists/perl/porters/305035
The RFC was a product of an observation in irc.perl.org #p5p that most people in the Perl community\, when asked for help with threads in Perl\, will harshly denounce their viability and recommend that the problematic program be gutted and rewritten with almost anything but.
The #p5p IRC discussion began on Feb 19\, 2014\, with "Perl's threads are socially weird. They exist and are supported\, but they (and their users) are heavily deprecated in the community."
Or it began with an observation about 90 minutes earlier in a #p5p discussion with/about a Perl novice:
right\, because most channels don't answers threads questions because most channels are full of people who don't use threads.pm because 'use threads;' almost always indicates brain damage
-- Rocco Caputo \rcaputo@​pobox\.com
p5pRT
commented
8 years ago On Tue\, May 5\, 2015 at 1:49 PM\, Rocco Caputo \rcaputo@​pobox\.com wrote:
On May 5\, 2015\, at 11:23\, Eric Brine \perlbug\-followup@​perl\.org wrote:
# New Ticket Created by "Eric Brine" # Please include the string: [perl #125106] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125106 >
threads.pm says:
The use of interpreter-based threads in perl is officially discouraged.
Why is that? Is it just because they're rather heavy? If so\, why not just say so and let the user decide whether he wants to use them or not? After all\, heavy threads aren't a problem if you reuse them (as in a worker model).
If your documentation is anything like mine\, the paragraph before that statement tries to explain why. Perhaps the discouragement should begin with "Because of this\, the use of...." to hint that the reader may have skipped something important and to discourage people from taking a provocative statement out of context.
One might think that\, except that the word "discouraged" is linked to a definition that means something quite different than "not to be used without understanding what it means that Perl threads are heavy". It actually defines "discouraged" as "we'd like to get rid of them\, but we're not doing it now"
- *discouraged*
From time to time\, we may mark language constructs and features which we consider to have been mistakes as *discouraged*. Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core.
p5pRT
commented
8 years ago On 05/10/2016 10:22 PM\, Eric Brine wrote:
On Tue\, May 5\, 2015 at 1:49 PM\, Rocco Caputo \<rcaputo@pobox.com \mailto​:rcaputo@​pobox\.com>wrote:
> On May 5\, 2015\, at 11​:23\, Eric Brine \<perlbug\-followup@​perl\.org \<mailto​:perlbug\-followup@​perl\.org>> wrote​: > > \# New Ticket Created by "Eric Brine" > \# Please include the string​: \[perl \#125106\] > \# in the subject line of all future correspondence about this issue\. > \# \<URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125106 \<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125106>> > > > threads\.pm \<http​://threads\.pm>says​: > > The use of interpreter\-based threads in perl is officially discouraged\. > > > Why is that? Is it just because they're rather heavy? If so\, why not just > say so and let the user decide whether he wants to use them or not? After > all\, heavy threads aren't a problem if you reuse them \(as in a worker > model\)\. If your documentation is anything like mine\, the paragraph before that statement tries to explain why\. Perhaps the discouragement should begin with "Because of this\, the use of\.\.\.\." to hint that the reader may have skipped something important and to discourage people from taking a provocative statement out of context\.One might think that\, except that the word "discouraged" is linked to a definition that means something quite different than "not to be used without understanding what it means that Perl threads are heavy". It actually defines "discouraged" as "we'd like to get rid of them\, but we're not doing it now"
* *discouraged*
From time to time\, we may mark language constructs and features which we consider to have been mistakes as \*discouraged\*\. Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core\.
I agree this is a confusing description.
I also like Rocco's comments on describing *why* something is the way it is\, even if briefly. I found an example in the thread Rocco linked[1]\, by Aristotle\, on how to explain things briefly and clearly:
"complex data structures are hard to share properly"
The quote is out of context and does not constitute as the patch I would envision\, but it's an example of how to describe something complicated succinctly. Perhaps we can form a patch describing what we mean by "discouraged". Having context might excuse us from trying to find a better term for this than "discouraged".
[1] Thanks\, Rocco!
p5pRT
commented
8 years ago On Wed\, 11 May 2016 19:54:37 +0200 Sawyer X \xsawyerx@​gmail\.com wrote:
I agree this is a confusing description.
I also like Rocco's comments on describing *why* something is the way it is\, even if briefly. I found an example in the thread Rocco linked[1]\, by Aristotle\, on how to explain things briefly and clearly:
"complex data structures are hard to share properly"
The quote is out of context and does not constitute as the patch I would envision\, but it's an example of how to describe something complicated succinctly. Perhaps we can form a patch describing what we mean by "discouraged". Having context might excuse us from trying to find a better term for this than "discouraged".
[1] Thanks\, Rocco!
If we have some better wording I could put it into perlconcurrency.pod as well...
-- Paul "LeoNerd" Evans
leonerd@leonerd.org.uk http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS
p5pRT
commented
6 years ago The documentation of threads.pm says that perl threads are "discouraged". This causes lots of confusion\, as the term is defined in perlpolicy in the following way:
discouraged From time to time\, we may mark language constructs and features which we consider to have been mistakes as discouraged. Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core.
See also #125106\, which seems to lead nowhere\, maybe because it was too ambitious. This patch provides just the minimal change.
See also http://www.perlmonks.org/?node_id=1107534 .
p5pRT
commented
6 years ago p5pRT
commented
6 years ago On Sat\, Mar 24\, 2018 at 12:31 PM\, E. Choroba \perlbug\-followup@​perl\.org wrote:
# New Ticket Created by "E. Choroba" # Please include the string: [perl #133021] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133021 >
----------------------------------------------------------------- [Please describe your issue here]
The documentation of threads.pm says that perl threads are "discouraged". This causes lots of confusion\, as the term is defined in perlpolicy in the following way:
discouraged From time to time\, we may mark language constructs and features which we consider to have been mistakes as discouraged. Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core.
See also #125106\, which seems to lead nowhere\, maybe because it was too ambitious. This patch provides just the minimal change.
See also http://www.perlmonks.org/?node_id=1107534 .
'not recommended' seems too lenient. 'strongly recommended against'?
-Dan
p5pRT
commented
6 years ago The RT System itself - Status changed from 'new' to 'open'
p5pRT
commented
6 years ago On Sat\, 24 Mar 2018 09:31:06 -0700\, choroba@matfyz.cz wrote:
From time to time\, we may mark language constructs and features which we consider to have been mistakes as discouraged\.
Threads in their current implementation are arguable a mistake. You loose ~30% of performance and who-measured-how-much memory by just using perl build with threads. And most of XS CPAN is up to some degree threads-unsafe (pure-perl CPAN is safe\, so the goals for threads is reached\, but still).
Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core\.
Threads are not going anywhere by this definition. Just discouraged.
p5pRT
commented
6 years ago On Sun\, Mar 25\, 2018 at 04:20:13PM -0700\, Sergey Aleynikov via RT wrote:
Threads in their current implementation are arguable a mistake. You loose ~30% of performance and who-measured-how-much memory by just using perl build with threads.
The test suite on blead on my system runs 17% slower under ithreads\, but that includes running a bunch of ithreads-only tests too. So I think 30% is overstating it.
I'm not aware that a normal single-threaded process uses significantly more memory under ithreads.
-- Hofstadter's Law: It always takes longer than you expect\, even when you take into account Hofstadter's Law.
p5pRT
commented
6 years ago Since this change doesn't have a ticket attached to it\, i figured it would be useful to provide the context of what led to its creation.
This discouragement was introduced by rjbs in commit 10a4597703f4b044d3f968bf3923644ea2387f5c. I've extracted some of the chat log that led to it.
https://gist.github.com/wchristian/97525932a89d0785e07371c60d84209c
And here's the p5p mailing list thread that followed afterwards:
https://www.nntp.perl.org/group/perl.perl5.porters/2014/03/msg213382.html
It's a lot of material\, but i hope it'll help understand the context here.
p5pRT
commented
6 years ago On Mon\, 26 Mar 2018 01:36:54 -0700\, davem wrote:
The test suite on blead on my system runs 17% slower under ithreads\, but that includes running a bunch of ithreads-only tests too. So I think 30% is overstating it.
It depends on the operation type. I've measured hash accesses and got ~10% loss on threaded build for 5.26.1.
p5pRT
commented
6 years ago On Mon\, Mar 26\, 2018 at 04:52:54AM -0700\, Sergey Aleynikov via RT wrote:
On Mon\, 26 Mar 2018 01:36:54 -0700\, davem wrote:
The test suite on blead on my system runs 17% slower under ithreads\, but that includes running a bunch of ithreads-only tests too. So I think 30% is overstating it.
It depends on the operation type. I've measured hash accesses and got ~10% loss on threaded build for 5.26.1.
10% isn't 30%.
-- A walk of a thousand miles begins with a single step... then continues for another 1\,999\,999 or so.
p5pRT
commented
6 years ago As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
Just my two cents.
p5pRT
commented
6 years ago As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
Just my two cents.
p5pRT
commented
6 years ago On 03/26/2018 06:54 PM\, Jerry D. Hedden via RT wrote:
As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
Just my two cents.
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
Even a 30% slowdown will be fully acceptable if you can divide the work up into 8 or 16 parallel pieces. The gain far outweighs the cost.
p5pRT
commented
6 years ago On Mon\, Mar 26\, 2018 at 8:54 PM\, Jerry D. Hedden via RT \< perlbug-comment@perl.org> wrote:
As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
Just my two cents.
While all of this is true\, the other audience for this documentation section is the new Perl user who thinks threads sound like a nice lightweight way to spread work around without breaking all of their code (because most of the rest of Perl works this way\, you can just throw things together and they all work). These users do not read far enough to realize that threads are not the optimal solution to their requirements without a strong warning. In my opinion\, severe discouragement (if not the perlpolicy defined term) is warranted; if someone can achieve this while still making it clear that threads are not going to be deprecated for valid and optimal use cases for them\, this would be ideal.
-Dan
p5pRT
commented
6 years ago On Mon\, Mar 26\, 2018 at 07:20:29PM -0600\, Karl Williamson wrote:
On 03/26/2018 06:54 PM\, Jerry D. Hedden via RT wrote:
As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
Just my two cents.
+1
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
+1
-- The Enterprise's efficient long-range scanners detect a temporal vortex distortion in good time\, allowing it to be safely avoided via a minor course correction. -- Things That Never Happen in "Star Trek" #21
p5pRT
commented
6 years ago On Mar 27\, 2018\, at 07:15\, Dave Mitchell \davem@​iabyn\.com wrote:
On Mon\, Mar 26\, 2018 at 07:20:29PM -0600\, Karl Williamson wrote:
On 03/26/2018 06:54 PM\, Jerry D. Hedden via RT wrote:
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
I started the discussion that led to the addition of the discouragement warning. Its original purpose was to manage users' expectations of the kind of help they'd get from the larger Perl community for trying to use this feature.
Toning down the notice doesn't break its original purpose.
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
Getting rid of the notice entirely would resume failing to manage users' expectations.
In case someone on-list isn't aware of the advice given off-list\, I'm closing with a small list of quotes from freenode #perl. These are pieces of real advice given to Perl users looking for help using threads. Some of you may recognize things you've said.
Some of you may be tempted to say that IRC isn't representative of a larger perl community\, but many of the vocal opponents on IRC also participate in other areas. They use and advocate Perl professionally. They attend or speak at Perl conferences. They are perl5 porters.
This list includes both old and recent statements:
don't use threads
When someone says "I need to use threads" in perl\, they (99.9% of the time) mean "I don't know what I'm talking about\, please (re-)educate me"
But at least you didn't start out trying to use threads.
Yes.. Don't use threads. At all.
The simple answer is "don't use threads." Just fork if you must.
no\, because threads are usually a stupid answer.
do not use threads.
'use threads' is "please let me use the windows fork hack code"
also\, don't use threads in Perl; if your problem requires threads\, don't use Perl
you seem to have missed the point earlier that was made about "dont use threads"
the thread lecture is don't use threads in Perl\, or don't use Perl
before anything else\, notice that people here will start telling you about how you probably shouldn't use threads in Perl.
Hi. Please don't use threads.
I wonder how many times we've told qubit not to use threads and how mayn times they've ignored us.
Because threads are not what you want. Threads in Perl\, doubly so
(Almost) Everyone who wants to use threads in perl understands them incorrectly.
very few professional perl devs will use threads for real code
because threads are a stupid way to do concurrency in perl.
I really wish there was this little genie like the little green dude in The Flintstones that would pop up and educate people when they think they want to use threads in perl.
Don't use threads\, would be our suggestion
If you want to wrap your head in crazy concurrency bugs and random failures\, then use threads.
Suggestion: don't use threads
don't use threads in Perl
perlbot: threads is Friends don't let friends use threads ; Elizabeth on Threads - http://www.perlmonks.org/index.pl?node_id=288022
basically: stop trying to use threads.
and really you don't want to use threads at all
or you can use threads\, and we will ignore you when it all breaks.
step 1: don't use threads
The response is don't use threads
never use threads in perl
So then they're always asking "how can I do threads" and we say "don't use threads" and they complain and say "waaaaa I need them for doing lots of things at once" and then we say "no you dno't\, you can do other things".. and they give a look of complete utter confusion.
yes but in your case it is because threads are known to be broken and so will cause you problems
First convince your department not to use threads.
I don't imagine you'll get much help.. Our advice is "don't use threads"... hardly anyone here does much if anything with threads.. So likely nobody here really knows
This may be the definitive idea of why allowing people to use threads is a bad idea.
you should not use threads.
don't use threads. problem solved.
don't use threads is the answer
You don't want to use threads.
ah\, the truth "I use threads because I'm too dumb to understand select"
But if you're wanting to use threads\, odds are you are doing something wrong :)
the answer is invariably "don't use threads" :)
ah\, well\, your first mistake was to actually seriously use threads in Perl
look\, we can't actually advise you on a concurrency solution unless you tell us what the program's going to do - although the odds are pretty good that we'll say "don't use threads" since that's almost always the worst choice
code containing 'use Switch' or 'use threads' can be assumed to be broken
the best advice I can give is to not use threads in Perl
some idiot did 'use threads;' and now you have a broken program that needs replacing
DON'T USE THREADS\, DON'T TELL OTHERS TO USE THREADS.
also\, #perl will almost always refuse to help with threads. because threads in perl are shit.
-- Rocco Caputo \rcaputo@​pobox\.com
p5pRT
commented
6 years ago Perhaps a reference to the âforksâ module on CPAN could be added?
https://metacpan.org/pod/forks
Liz
p5pRT
commented
6 years ago On Tue\, Mar 27\, 2018 at 3:20 AM\, Karl Williamson \public@​khwilliamson\.com wrote:
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
I think the main problem with the current wording is that it discourages without explaining why. In particular it doesn't explain that threads.pm does something different than what many people expect it to do.
People expect it to be good for "share all the data" scenarios (because pretty much anything else called threads is)\, even though it is terrible at that. This confusion isn't helped by threads implementations in commonly used implementations of languages occupying the same niches as we do (e.g. cpython\, Ruby MRI) sucking in exactly the opposite way (they're good at sharing but useless at actually being parallel).
It's not so much threads.pm that needs a big fat warnings\, it's threads::shared that does.
Leon
p5pRT
commented
6 years ago On Mon\, 26 Mar 2018 06:06:47 -0700\, davem wrote:
10% isn't 30%.
My idea was to show that you can't attribute all the test run slowdown to additional threading tests. Probably\, I should've made this clearer in that message. As for the total slowdown figures\, it depends on the application\, but it's still "free" - you get it even if you don't start any threads.
p5pRT
commented
6 years ago On Mon\, 26 Mar 2018 18:21:02 -0700\, public@khwilliamson.com wrote:
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
Even a 30% slowdown will be fully acceptable if you can divide the work up into 8 or 16 parallel pieces. The gain far outweighs the cost.
But if you use processes for parallelism and not threads\, you don't get this slowdown at all and get the same multicore boost. And you don't have to guess which XS module will segfault next. The only case when I find using threads reasonable is writing GUI\, where you _have to_ be in the same address space.
p5pRT
commented
6 years ago On 2018-03-27\, Elizabeth Mattijsen \liz@​dijkmat\.nl wrote:
Perhaps a reference to the âforksâ module on CPAN could be added?
Which does not work either \https://rt.cpan.org/Public/Bug/Display.html?id=123248 (because of thread-safety fixes in File::Glob).
-- Petr
p5pRT
commented
6 years ago On 28 Mar 2018\, at 12:36\, Petr Pisar \ppisar@​redhat\.com wrote: On 2018-03-27\, Elizabeth Mattijsen \liz@​dijkmat\.nl wrote:
Perhaps a reference to the âforksâ module on CPAN could be added? Which does not work either \https://rt.cpan.org/Public/Bug/Display.html?id=123248 (because of thread-safety fixes in File::Glob).
Way back when in 2002 / 2003\, I had some free time on my hands and decided to see how I could use that to the advancement of Perl. The 5.8.0 release was still in the works\, and it had some fine new features. One of them being ithreads.
Having always been very interested in executing stuff asynchronously / in parallel\, I decided to cut my teeth into it\, coming from having been exposed to pthreads in C.
This resulted in quite a number of Thread:: modules on CPAN [1]. When I finally found out *how* Perl ithreads were implemented\, I was *very* disappointed.
Having become considered a bit of a âithreadsâ wizard on PerlMonks at the time\, I basically explained the situation *almost 15 years ago* [2]
Apart from bug fixes\, I donât think the situation since then has changed significantly.
So\, back to the documentation issue. Perhaps the âPerformance considerationsâ paragraph from that post could serve as inspiration.
Liz
[1] https://metacpan.org/author/ELIZABETH [2] http://www.perlmonks.org/?node_id=288022
p5pRT
commented
6 years ago Talking about PerlMonks\, try reading some of the posts from BrowserUk\, the current threads "wizard" there.
http://www.perlmonks.org/?node_id=3989;BIT=threads;a=BrowserUk
On Wed\, 28 Mar 2018 04:04:44 -0700\, elizabeth wrote:
On 28 Mar 2018\, at 12:36\, Petr Pisar \ppisar@​redhat\.com wrote: On 2018-03-27\, Elizabeth Mattijsen \liz@​dijkmat\.nl wrote:
Perhaps a reference to the âforksâ module on CPAN could be added? Which does not work either \https://rt.cpan.org/Public/Bug/Display.html?id=123248 (because of thread-safety fixes in File::Glob).
Way back when in 2002 / 2003\, I had some free time on my hands and decided to see how I could use that to the advancement of Perl. The 5.8.0 release was still in the works\, and it had some fine new features. One of them being ithreads.
Having always been very interested in executing stuff asynchronously / in parallel\, I decided to cut my teeth into it\, coming from having been exposed to pthreads in C.
This resulted in quite a number of Thread:: modules on CPAN [1]. When I finally found out *how* Perl ithreads were implemented\, I was *very* disappointed.
Having become considered a bit of a âithreadsâ wizard on PerlMonks at the time\, I basically explained the situation *almost 15 years ago* [2]
Apart from bug fixes\, I donât think the situation since then has changed significantly.
So\, back to the documentation issue. Perhaps the âPerformance considerationsâ paragraph from that post could serve as inspiration.
Liz
[1] https://metacpan.org/author/ELIZABETH [2] http://www.perlmonks.org/?node_id=288022
p5pRT
commented
6 years ago * E. Choroba \perlbug\-followup@​perl\.org [2018-03-25 21:31]:
See also #125106\, which seems to lead nowhere\, maybe because it was too ambitious. This patch provides just the minimal change.
The minimal change would be to simply remove the sentence\, and that is what I would favour to begin with.
* Christian Walde \walde\.christian@​gmail\.com [2018-03-26 12:49]:
Since this change doesn't have a ticket attached to it\, i figured it would be useful to provide the context of what led to its creation.
Thank you. I vaguely remembered having been a part of that discussion and am glad to have the context back; I am also relieved to see that my current position is close to my previous stance.
This discouragement was introduced by rjbs in commit 10a4597703f4b044d3f968bf3923644ea2387f5c. I've extracted some of the chat log that led to it.
I regret not catching this at the time\, but the reason given for calling the feature discouraged does not meet the technical definition of the term in perlpolicy. So the statement in the threads.pm POD is simply not true.
I must admit my own anti-ithreads bias here\, and that this is probably what led me to skip over that issue. I am not unhappy to see threads discouraged-in-the-colloquial-sense and have always avoided them myself (with success).
However\, for some people they are the only option.
And they are not ever being removed from Perl\, even as a far-future possibility. They are thus not discouraged by the definition of that term in perlpolicy. However contentious everything else may be\, that much is unambiguously true.
* Jerry D. Hedden via RT \perlbug\-comment@​perl\.org [2018-03-27 02:59]:
As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
Ditto.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use.
Ditto.
I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
Ditto.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
Here we somewhat depart. To me the main point is that some people some of the time have no alternative to ithreads. And so if you need to use them\, well then you need to use them.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
The discouragement notice just belongs deleted. Iâm not sure that the heading would be too strong without it so Iâm -0 on changing that.
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
+1
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
This may depend on perspective. If I were in a situation without the choice of something else\, I could imagine finding them VERY useful. :-)
* Leon Timmermans \fawaka@​gmail\.com [2018-03-28 02:30]:
I think the main problem with the current wording is that it discourages without explaining why.
There is a long list of appropriately neutral explanation of footguns\, but only miles down the page under BUGS AND LIMITATIONS. There has to be a reference to that.
In particular it doesn't explain that threads.pm does something different than what many people expect it to do.
People expect it to be good for "share all the data" scenarios (because pretty much anything else called threads is)\, even though it is terrible at that. This confusion isn't helped by threads implementations in commonly used implementations of languages occupying the same niches as we do (e.g. cpython\, Ruby MRI) sucking in exactly the opposite way (they're good at sharing but useless at actually being parallel).
It's not so much threads.pm that needs a big fat warnings\, it's threads::shared that does.
+1
I would be in favour of a second patch which adds this up top\, which can be bikeshedded if people feel strongly enough about it. I am willing to propose new language for that.
First up Iâd want to remove the untrue statement\, however.
* Rocco Caputo \rcaputo@​pobox\.com [2018-03-27 16:35]:
I started the discussion that led to the addition of the discouragement warning. Its original purpose was to manage users' expectations of the kind of help they'd get from the larger Perl community for trying to use this feature. [âŠ] Getting rid of the notice entirely would resume failing to manage users' expectations.
The notice was simply incorrect and therefore could not possibly manage usersâ expectations correctly. If the fact that youâre talking about is that using threads will get you yelled at by some\, then how is any user expectation managed correctly by *not* telling the user that and instead just telling them to not use threads?
In case someone on-list isn't aware of the advice given off-list\, I'm closing with a small list of quotes from freenode #perl. These are pieces of real advice given to Perl users looking for help using threads. Some of you may recognize things you've said.
How heavy is the tail on the long tail graph of who these quotes came from? How many counterexamples of users getting real help with threads are there?
Some of you may be tempted to say that IRC isn't representative of a larger perl community\, but many of the vocal opponents on IRC also participate in other areas. They use and advocate Perl professionally. They attend or speak at Perl conferences. They are perl5 porters.
Are you aware that what comes after your âbutâ doesnât rebut what comes before it? :-)
* Sergey Aleynikov via RT \perlbug\-followup@​perl\.org [2018-03-26 01:29]:
On Sat\, 24 Mar 2018 09:31:06 -0700\, choroba@matfyz.cz wrote:
From time to time\, we may mark language constructs and features which we consider to have been mistakes as discouraged\.Threads in their current implementation are arguable a mistake. You loose ~30% of performance and who-measured-how-much memory by just using perl build with threads. And most of XS CPAN is up to some degree threads-unsafe (pure-perl CPAN is safe\, so the goals for threads is reached\, but still).
Any feasible implementation would be a mistake of a different kind. So faced with the problem of needing some kind of threads but having only bad choices for their implementation\, Perl made a different bad choice than other similar languages. (Cf. Leonâs reply.)
And no random scary numbers pulled out of a hat please. I donât mind mentioning that thread-supporting perls are slower at an above-noise level\, but if youâre going to quote specific numbers\, they must come with actual benchmarks to contextualise them. Just â30% slower!!1!â is FUD. (I myself compile my perls without threads\, because if I never ever use threads then why pay for them? That is an argument that holds even in absence of any exact figures.)
Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core\.Threads are not going anywhere by this definition.
Indeed.
Just discouraged.
In the colloquial sense maybe\, but as you found\, not in the perlpolicy sense.
* Sergey Aleynikov via RT \perlbug\-followup@​perl\.org [2018-03-28 03:04]:
On Mon\, 26 Mar 2018 18:21:02 -0700\, public@khwilliamson.com wrote:
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
Even a 30% slowdown will be fully acceptable if you can divide the work up into 8 or 16 parallel pieces. The gain far outweighs the cost.
But if you use processes for parallelism and not threads\, you don't get this slowdown at all and get the same multicore boost. And you don't have to guess which XS module will segfault next. The only case when I find using threads reasonable is writing GUI\, where you _have to_ be in the same address space.
Except some people do not have the option.
Regards\, -- Aristotle Pagaltzis // \<http://plasmasturm.org/>
p5pRT
commented
6 years ago On 03/29/2018 02:46 PM\, Aristotle Pagaltzis wrote:
* E. Choroba \perlbug\-followup@​perl\.org [2018-03-25 21:31]:
See also #125106\, which seems to lead nowhere\, maybe because it was too ambitious. This patch provides just the minimal change.
The minimal change would be to simply remove the sentence\, and that is what I would favour to begin with.
* Christian Walde \walde\.christian@​gmail\.com [2018-03-26 12:49]:
Since this change doesn't have a ticket attached to it\, i figured it would be useful to provide the context of what led to its creation.
Thank you. I vaguely remembered having been a part of that discussion and am glad to have the context back; I am also relieved to see that my current position is close to my previous stance.
This discouragement was introduced by rjbs in commit 10a4597703f4b044d3f968bf3923644ea2387f5c. I've extracted some of the chat log that led to it.
I regret not catching this at the time\, but the reason given for calling the feature discouraged does not meet the technical definition of the term in perlpolicy. So the statement in the threads.pm POD is simply not true.
I must admit my own anti-ithreads bias here\, and that this is probably what led me to skip over that issue. I am not unhappy to see threads discouraged-in-the-colloquial-sense and have always avoided them myself (with success).
However\, for some people they are the only option.
And they are not ever being removed from Perl\, even as a far-future possibility. They are thus not discouraged by the definition of that term in perlpolicy. However contentious everything else may be\, that much is unambiguously true.
* Jerry D. Hedden via RT \perlbug\-comment@​perl\.org [2018-03-27 02:59]:
As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message.
Ditto.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use.
Ditto.
I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD.
Ditto.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users.
Here we somewhat depart. To me the main point is that some people some of the time have no alternative to ithreads. And so if you need to use them\, well then you need to use them.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch).
The discouragement notice just belongs deleted. Iâm not sure that the heading would be too strong without it so Iâm -0 on changing that.
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated.
+1
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature.
This may depend on perspective. If I were in a situation without the choice of something else\, I could imagine finding them VERY useful. :-)
* Leon Timmermans \fawaka@​gmail\.com [2018-03-28 02:30]:
I think the main problem with the current wording is that it discourages without explaining why.
There is a long list of appropriately neutral explanation of footguns\, but only miles down the page under BUGS AND LIMITATIONS. There has to be a reference to that.
In particular it doesn't explain that threads.pm does something different than what many people expect it to do.
People expect it to be good for "share all the data" scenarios (because pretty much anything else called threads is)\, even though it is terrible at that. This confusion isn't helped by threads implementations in commonly used implementations of languages occupying the same niches as we do (e.g. cpython\, Ruby MRI) sucking in exactly the opposite way (they're good at sharing but useless at actually being parallel).
It's not so much threads.pm that needs a big fat warnings\, it's threads::shared that does.
+1
I would be in favour of a second patch which adds this up top\, which can be bikeshedded if people feel strongly enough about it. I am willing to propose new language for that.
First up Iâd want to remove the untrue statement\, however.
* Rocco Caputo \rcaputo@​pobox\.com [2018-03-27 16:35]:
I started the discussion that led to the addition of the discouragement warning. Its original purpose was to manage users' expectations of the kind of help they'd get from the larger Perl community for trying to use this feature. [âŠ] Getting rid of the notice entirely would resume failing to manage users' expectations.
The notice was simply incorrect and therefore could not possibly manage usersâ expectations correctly. If the fact that youâre talking about is that using threads will get you yelled at by some\, then how is any user expectation managed correctly by *not* telling the user that and instead just telling them to not use threads?
In case someone on-list isn't aware of the advice given off-list\, I'm closing with a small list of quotes from freenode #perl. These are pieces of real advice given to Perl users looking for help using threads. Some of you may recognize things you've said.
How heavy is the tail on the long tail graph of who these quotes came from? How many counterexamples of users getting real help with threads are there?
Some of you may be tempted to say that IRC isn't representative of a larger perl community\, but many of the vocal opponents on IRC also participate in other areas. They use and advocate Perl professionally. They attend or speak at Perl conferences. They are perl5 porters.
Are you aware that what comes after your âbutâ doesnât rebut what comes before it? :-)
* Sergey Aleynikov via RT \perlbug\-followup@​perl\.org [2018-03-26 01:29]:
On Sat\, 24 Mar 2018 09:31:06 -0700\, choroba@matfyz.cz wrote:
From time to time\, we may mark language constructs and features which we consider to have been mistakes as discouraged\.Threads in their current implementation are arguable a mistake. You loose ~30% of performance and who-measured-how-much memory by just using perl build with threads. And most of XS CPAN is up to some degree threads-unsafe (pure-perl CPAN is safe\, so the goals for threads is reached\, but still).
Any feasible implementation would be a mistake of a different kind. So faced with the problem of needing some kind of threads but having only bad choices for their implementation\, Perl made a different bad choice than other similar languages. (Cf. Leonâs reply.)
And no random scary numbers pulled out of a hat please. I donât mind mentioning that thread-supporting perls are slower at an above-noise level\, but if youâre going to quote specific numbers\, they must come with actual benchmarks to contextualise them. Just â30% slower!!1!â is FUD. (I myself compile my perls without threads\, because if I never ever use threads then why pay for them? That is an argument that holds even in absence of any exact figures.)
Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core\.Threads are not going anywhere by this definition.
Indeed.
Just discouraged.
In the colloquial sense maybe\, but as you found\, not in the perlpolicy sense.
* Sergey Aleynikov via RT \perlbug\-followup@​perl\.org [2018-03-28 03:04]:
On Mon\, 26 Mar 2018 18:21:02 -0700\, public@khwilliamson.com wrote:
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
Even a 30% slowdown will be fully acceptable if you can divide the work up into 8 or 16 parallel pieces. The gain far outweighs the cost.
But if you use processes for parallelism and not threads\, you don't get this slowdown at all and get the same multicore boost. And you don't have to guess which XS module will segfault next. The only case when I find using threads reasonable is writing GUI\, where you _have to_ be in the same address space.
Except some people do not have the option.
Regards\,
It turns out that #125106 also is about this topic\, and I have merged the two tickets. You might want to look at the discussion there.
p5pRT
commented
6 years ago [Top-posted]
I agree with pretty much every point Aristotle made here. I want to highlight some issues to move forward on:
* As Aristotle explains\, this definition of "discouraged" does not match what it means in perlpolicy. It might meet what we refer to in day-to-day communication\, but this creates a gap between that and what our policy states it is. This is important because our technical decisions must conform to the policy. We must then remove the "discouraged" label.
* There is still room on how to define this recommendation to avoid. I'm personally fond of the short-but-honest get-to-the-core descriptions\, such as "This might not be what you want" or "This will likely not work the way you expect it to" or "This isn't what you would this it is" or "Unless you know how this works\, you will want to stay clear of it." There are far better English speakers who could provide a nuanced phrasing and far better explanation of why this isn't what you wish\, technically.
* I think we definitely need to add more explanations (both on threads and threads::shared) for the difference in behavior vs. expected behavior\, allowing users to jump from short-and-sweet to the details\, helping them make a decision of whether they want to use threads.pm or not - and if so\, how.
* As Aristotle requested: We should not put numbers without benchmarks.
We can say "noticeably slower" or "noticeably slower for \
* It would also be good to do the counter of suggesting not to use threads (or pointing at the grain of salt you have to take when wanting to use it)\, which is to note when it *is* useful to use threads\, providing the narrow cases in which it shines or works the way you expect it to (or the way it's designed to).
Did I miss anything?
On 03/29/2018 11:46 PM\, Aristotle Pagaltzis wrote:
* E. Choroba \perlbug\-followup@​perl\.org [2018-03-25 21:31]:
See also #125106\, which seems to lead nowhere\, maybe because it was too ambitious. This patch provides just the minimal change. The minimal change would be to simply remove the sentence\, and that is what I would favour to begin with.
* Christian Walde \walde\.christian@​gmail\.com [2018-03-26 12:49]:
Since this change doesn't have a ticket attached to it\, i figured it would be useful to provide the context of what led to its creation. Thank you. I vaguely remembered having been a part of that discussion and am glad to have the context back; I am also relieved to see that my current position is close to my previous stance.
This discouragement was introduced by rjbs in commit 10a4597703f4b044d3f968bf3923644ea2387f5c. I've extracted some of the chat log that led to it. I regret not catching this at the time\, but the reason given for calling the feature discouraged does not meet the technical definition of the term in perlpolicy. So the statement in the threads.pm POD is simply not true.
I must admit my own anti-ithreads bias here\, and that this is probably what led me to skip over that issue. I am not unhappy to see threads discouraged-in-the-colloquial-sense and have always avoided them myself (with success).
However\, for some people they are the only option.
And they are not ever being removed from Perl\, even as a far-future possibility. They are thus not discouraged by the definition of that term in perlpolicy. However contentious everything else may be\, that much is unambiguously true.
* Jerry D. Hedden via RT \perlbug\-comment@​perl\.org [2018-03-27 02:59]:
As one of several people who have maintained the various threads modules over the years\, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message. Ditto.
The fact is that threads work\, they are maintained\, and they currently do not have any bugs preventing their use. Ditto.
I acknowledge that not all Perl modules are thread-safe\, but there is sufficient documentation to that affect in the POD. Ditto.
I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless\, threads are useful\, and are being used in the wild. (I\, for one\, have even used them to good effect in production code. \<Gasp!>) Yes\, if you don't know what you're doing\, threads can be problematic. ("There be dragons...\," and all that.) However\, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users. Here we somewhat depart. To me the main point is that some people some of the time have no alternative to ithreads. And so if you need to use them\, well then you need to use them.
I feel that\, while the wording of the POD notice is reasonable\, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE\, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch). The discouragement notice just belongs deleted. Iâm not sure that the heading would be too strong without it so Iâm -0 on changing that.
Since the addition of the 'discouraged' message\, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above\, namely that threads work (but you have to know what you're doing)\, and that threads have not been deprecated. +1
I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all\, Perl has always been about options. However\, there ARE Perl developers that do feel they are a VERY useful feature. This may depend on perspective. If I were in a situation without the choice of something else\, I could imagine finding them VERY useful. :-)
* Leon Timmermans \fawaka@​gmail\.com [2018-03-28 02:30]:
I think the main problem with the current wording is that it discourages without explaining why. There is a long list of appropriately neutral explanation of footguns\, but only miles down the page under BUGS AND LIMITATIONS. There has to be a reference to that.
In particular it doesn't explain that threads.pm does something different than what many people expect it to do.
People expect it to be good for "share all the data" scenarios (because pretty much anything else called threads is)\, even though it is terrible at that. This confusion isn't helped by threads implementations in commonly used implementations of languages occupying the same niches as we do (e.g. cpython\, Ruby MRI) sucking in exactly the opposite way (they're good at sharing but useless at actually being parallel).
It's not so much threads.pm that needs a big fat warnings\, it's threads::shared that does. +1
I would be in favour of a second patch which adds this up top\, which can be bikeshedded if people feel strongly enough about it. I am willing to propose new language for that.
First up Iâd want to remove the untrue statement\, however.
* Rocco Caputo \rcaputo@​pobox\.com [2018-03-27 16:35]:
I started the discussion that led to the addition of the discouragement warning. Its original purpose was to manage users' expectations of the kind of help they'd get from the larger Perl community for trying to use this feature. [âŠ] Getting rid of the notice entirely would resume failing to manage users' expectations. The notice was simply incorrect and therefore could not possibly manage usersâ expectations correctly. If the fact that youâre talking about is that using threads will get you yelled at by some\, then how is any user expectation managed correctly by *not* telling the user that and instead just telling them to not use threads?
In case someone on-list isn't aware of the advice given off-list\, I'm closing with a small list of quotes from freenode #perl. These are pieces of real advice given to Perl users looking for help using threads. Some of you may recognize things you've said. How heavy is the tail on the long tail graph of who these quotes came from? How many counterexamples of users getting real help with threads are there?
Some of you may be tempted to say that IRC isn't representative of a larger perl community\, but many of the vocal opponents on IRC also participate in other areas. They use and advocate Perl professionally. They attend or speak at Perl conferences. They are perl5 porters. Are you aware that what comes after your âbutâ doesnât rebut what comes before it? :-)
* Sergey Aleynikov via RT \perlbug\-followup@​perl\.org [2018-03-26 01:29]:
On Sat\, 24 Mar 2018 09:31:06 -0700\, choroba@matfyz.cz wrote:
From time to time\, we may mark language constructs and features which we consider to have been mistakes as discouraged\.Threads in their current implementation are arguable a mistake. You loose ~30% of performance and who-measured-how-much memory by just using perl build with threads. And most of XS CPAN is up to some degree threads-unsafe (pure-perl CPAN is safe\, so the goals for threads is reached\, but still). Any feasible implementation would be a mistake of a different kind. So faced with the problem of needing some kind of threads but having only bad choices for their implementation\, Perl made a different bad choice than other similar languages. (Cf. Leonâs reply.)
And no random scary numbers pulled out of a hat please. I donât mind mentioning that thread-supporting perls are slower at an above-noise level\, but if youâre going to quote specific numbers\, they must come with actual benchmarks to contextualise them. Just â30% slower!!1!â is FUD. (I myself compile my perls without threads\, because if I never ever use threads then why pay for them? That is an argument that holds even in absence of any exact figures.)
Discouraged features aren't currently candidates for removal\, but we may later deprecate them if they're found to stand in the way of a significant improvement to the Perl core\.Threads are not going anywhere by this definition. Indeed.
Just discouraged. In the colloquial sense maybe\, but as you found\, not in the perlpolicy sense.
* Sergey Aleynikov via RT \perlbug\-followup@​perl\.org [2018-03-28 03:04]:
On Mon\, 26 Mar 2018 18:21:02 -0700\, public@khwilliamson.com wrote:
I agree with the above. And I really don't like 'not recommended' as I think it is too strong. Maybe just list the problems
Even a 30% slowdown will be fully acceptable if you can divide the work up into 8 or 16 parallel pieces. The gain far outweighs the cost. But if you use processes for parallelism and not threads\, you don't get this slowdown at all and get the same multicore boost. And you don't have to guess which XS module will segfault next. The only case when I find using threads reasonable is writing GUI\, where you _have to_ be in the same address space. Except some people do not have the option.
Regards\,
p5pRT
commented
6 years ago On Apr 5\, 2018\, at 04:39\, Sawyer X \xsawyerx@​gmail\.com wrote:
Did I miss anything?
I think so. The only part of the existing paragraph that tries to warn the reader about the difference between "supported" and "you will receive support":
"Few people ... will be able to provide help."
Even that is an understatement. Maybe it should say "willing or able".
In the context of the discussions leading up to the change\, telling people that they won't get much help is the most important part of that paragraph. The rest of it merely attempts to justify why it has to be said.
-- Rocco Caputo \rcaputo@​pobox\.com
p5pRT
commented
6 years ago On 04/05/2018 04:33 PM\, Rocco Caputo wrote:
On Apr 5\, 2018\, at 04:39\, Sawyer X \<xsawyerx@gmail.com \mailto​:xsawyerx@​gmail\.com> wrote:
Did I miss anything?
I think so. Â The only part of the existing paragraph that tries to warn the reader about the difference between "supported" and "you will receive support":
"Few people ... will be able to provide help."
Even that is an understatement. Â Maybe it should say "willing or able".
In the context of the discussions leading up to the change\, telling people that they won't get much help is the most important part of that paragraph. Â The rest of it merely attempts to justify why it has to be said.
You're right. That is a good point.
I was thinking about this "discouraged" thing more and thought maybe a phrasing of\, "Unless you're [intimately] familiar with how this module achieves threaded behavior\, it is ill-advised to use it." or something that suggests "You should probably understand this if you want to use it." We could add that without knowledge\, it can be unwieldy or something.
Jerry\, if you could also help us pin down a description that is not completely alarmist on the other\, but is honest about the inherent risks this adds to anyone who does not understand how it works - that would be great.
p5pRT
commented
6 years ago There's one thing you're missing and why i think it's bad to call it alarmist. It is a simple matter if fact that even if theoretically the threads implementation is bug free and the foot guns documented\, actually using threads often surfaces foot guns as heisenbugs:
- perl threads can work fine under a certain load\, but crash randomly once the load becomes too big - perl threads can work fine on one computer\, but crash randomly on another due to slight module differences
This leads to a situation where the most accurate way to describe threads is:
Threads are fine to use in production only once the people working with them know and understand exactly and perfectly how they are implemented\, what the drawbacks are\, and how to handle problems arising from cpan modules that were not written by such people or have mistakes regarding threads in them.
On Thu\, Apr 5\, 2018\, 10:40 Sawyer X \xsawyerx@​gmail\.com wrote:
Did I miss anything?
p5pRT
commented
6 years ago On Sun\, Apr 08\, 2018 at 10:23:20PM +0000\, Christian Walde wrote:
- perl threads can work fine under a certain load\, but crash randomly once the load becomes too big - perl threads can work fine on one computer\, but crash randomly on another due to slight module differences
Is this based on the idea that the ithreads implementation itself is inherently buggy\, or that some modules may not be thread-safe.
I'd be surprised if the former\, and if the latter\, surely that applies to any "real" threading implementation in any programming language?
-- Modern art: "That's easy\, I could have done that!" "Ah\, but you didn't!"
p5pRT
commented
6 years ago On 04/09/2018 10:51 AM\, Dave Mitchell wrote:
On Sun\, Apr 08\, 2018 at 10:23:20PM +0000\, Christian Walde wrote:
- perl threads can work fine under a certain load\, but crash randomly once the load becomes too big - perl threads can work fine on one computer\, but crash randomly on another due to slight module differences Is this based on the idea that the ithreads implementation itself is inherently buggy\, or that some modules may not be thread-safe.
I'd be surprised if the former\, and if the latter\, surely that applies to any "real" threading implementation in any programming language?
Could you please expand on this\, Dave?
p5pRT
commented
6 years ago On Mon\, Apr 09\, 2018 at 12:18:10PM +0300\, Sawyer X wrote:
On 04/09/2018 10:51 AM\, Dave Mitchell wrote:
On Sun\, Apr 08\, 2018 at 10:23:20PM +0000\, Christian Walde wrote:
- perl threads can work fine under a certain load\, but crash randomly once the load becomes too big - perl threads can work fine on one computer\, but crash randomly on another due to slight module differences Is this based on the idea that the ithreads implementation itself is inherently buggy\, or that some modules may not be thread-safe.
I'd be surprised if the former\, and if the latter\, surely that applies to any "real" threading implementation in any programming language?
Could you please expand on this\, Dave?
1) I think the perl ithreads implementation is fairly bug-free and stable these days.
2) If one's threaded code randomly craps out due to a 3rd party module being non-thread-safe\, or supposedly thread-safe but buggy\, then that's an issue with the third-party module. In principle any programming language will struggle with threads if used with a 3rd-party library that isn't thread-safe.
Perl's threads are supposedly safe at the perl language level. For example\, if two threads simultaneously try to push a value to the same array\, then a) the internal representation of the array (AV etc) won't be corrupted; b) both elements will get pushed; c) perl won't guarantee which order they're pushed in.
This is a lot better guarantee than threads in C for example.
The main danger area is with XS modules; pure perl modules should at least not corrupt the interpreter state; whether they work correctly (e.g. two threads both trying to update the same data file) is down to whether the module's author does the correct locking etc.
-- No matter how many dust sheets you use\, you will get paint on the carpet.
p5pRT
commented
6 years ago On 9 Apr 2018\, at 12:37\, Dave Mitchell \davem@​iabyn\.com wrote: On Mon\, Apr 09\, 2018 at 12:18:10PM +0300\, Sawyer X wrote:
Could you please expand on this\, Dave? Perl's threads are supposedly safe at the perl language level. For example\, if two threads simultaneously try to push a value to the same array\, then a) the internal representation of the array (AV etc) won't be corrupted; b) both elements will get pushed;
Is that something that was fixed in the past 10 years or so? I distinctly remember this only being true for *shared* arrays\, which involve a lot of overhead.
OOC\, is this also true for hashes to which keys are being added by several threads?
Also OOC\, if multiple threads increase / decrease the reference count of something\, is that also threadsafe by default? AKA\, will you never get double frees (increment missed) or DESTROY never getting called (decrement missed)?
FWIW\, I was discussing these things the other day at the German Perl Workshop\, so this caught my eye having thought about these things for the first time in a long while.
Liz
p5pRT
commented
6 years ago (missed out Ccing p5p)
----- Forwarded message from Dave Mitchell \davem@​iabyn\.com -----
Date: Mon\, 9 Apr 2018 15:59:14 +0100 From: Dave Mitchell \davem@​iabyn\.com To: Elizabeth Mattijsen \liz@​dijkmat\.nl Subject: Re: [perl #133021] Removed the word "discouraged" from threads' documentation Message-ID: \20180409145914\.GN14242@​iabyn\.com
On Mon\, Apr 09\, 2018 at 02:34:21PM +0200\, Elizabeth Mattijsen wrote:
Is that something that was fixed in the past 10 years or so? I distinctly remember this only being true for *shared* arrays\, which involve a lot of overhead.
Well\, non-shared arrays can only be accessed by a single thread\, so they are not an issue.
Running the following code\, which has 100 threads pushing and popping things off a shared array shows that it ends with the same number of elements as when it started.
use threads; use threads::shared;
my @a : shared = (1..100);
sub inc { for (1..10_000) { push @a\, $_; pop @a; } }
my @t; push @t\, threads->new(\&inc) for 1..100; $_->join for @t; printf "size=%d\n"\, scalar(@a);
I've run this code on 5.10.1 and 5.27.10 and in both cases it finishes with 100 elements.
OOC\, is this also true for hashes to which keys are being added by several threads?
(I don't know what OOC stands for).
If the different threads are using disjoint sets of keys when accessing the same shared hash\, they won't interfere with each other. If multiple threads are adding\, modifying and deleting the same key in a shared hash\, then of course the final result for that key in the hash will depend on the timing; but it won't be corrupt.
Also OOC\, if multiple threads increase / decrease the reference count of something\, is that also threadsafe by default? AKA\, will you never get double frees (increment missed) or DESTROY never getting called (decrement missed)?
The whole point of ithreads is that each thread gets its own interpreter\, with its own copy of every SV. So it's not possible for multiple threads to access\, let alone modify\, the reference count of an SV. threads::shared uses locking behind the scenes to mediate access to shared variables\, so again\, ref-counting isn't an issue.
-- "You're so sadly neglected\, and often ignored. A poor second to Belgium\, When going abroad." -- Monty Python\, "Finland"
----- End forwarded message -----
-- Please note that ash-trays are provided for the use of smokers\, whereas the floor is provided for the use of all patrons. -- Bill Royston
p5pRT
commented
6 years ago On 9 Apr 2018\, at 17:00\, Dave Mitchell \davem@​iabyn\.com wrote: On Mon\, Apr 09\, 2018 at 02:34:21PM +0200\, Elizabeth Mattijsen wrote:
Is that something that was fixed in the past 10 years or so? I distinctly remember this only being true for *shared* arrays\, which involve a lot of overhead. Well\, non-shared arrays can only be accessed by a single thread\, so they are not an issue.
Ah\, ok\, so youâre not talking OS-threads\, but perl ithreads.
My point was about Perl code running in separate OS-threads\, what I understand (perhaps wrongly) with things like Future. So inside of a single interpreter.
Iâm quite aware how Perl ithreads work\, for the past 15 years or so at least:
http://www.perlmonks.org/?node_id=288022
Liz
p5pRT
commented
6 years ago On Mon\, Apr 9\, 2018 at 12:05 PM\, Elizabeth Mattijsen \liz@​dijkmat\.nl wrote:
On 9 Apr 2018\, at 17:00\, Dave Mitchell \davem@​iabyn\.com wrote: On Mon\, Apr 09\, 2018 at 02:34:21PM +0200\, Elizabeth Mattijsen wrote:
Is that something that was fixed in the past 10 years or so? I distinctly remember this only being true for *shared* arrays\, which involve a lot of overhead. Well\, non-shared arrays can only be accessed by a single thread\, so they are not an issue.
Ah\, ok\, so youâre not talking OS-threads\, but perl ithreads.
My point was about Perl code running in separate OS-threads\, what I understand (perhaps wrongly) with things like Future. So inside of a single interpreter.
Iâm quite aware how Perl ithreads work\, for the past 15 years or so at least:
http​://www\.perlmonks\.org/?node\_id=288022Liz
Future has nothing to do with threading of any sort (though it could be leveraged in combination\, such as what IO::Async::Function does on Windows). It's a mechanism for cooperative multitasking. I'm not sure what OS threads you are talking about that would run "inside of a single interpreter." pthreads run outside the interpreter.
-Dan
p5pRT
commented
6 years ago On 9 Apr 2018\, at 19:00\, Dan Book \grinnz@​gmail\.com wrote: On Mon\, Apr 9\, 2018 at 12:05 PM\, Elizabeth Mattijsen \liz@​dijkmat\.nl wrote:
My point was about Perl code running in separate OS-threads\, what I understand (perhaps wrongly) with things like Future. So inside of a single interpreter. Future has nothing to do with threading of any sort (though it could be leveraged in combination\, such as what IO::Async::Function does on Windows). It's a mechanism for cooperative multitasking. I'm not sure what OS threads you are talking about that would run "inside of a single interpreter." pthreads run outside the interpreter.
Ok\, so if I understand you correctly: if you run some pthreads outside of a Perl interpreter thread\, you will never be able to run any Perl code in that pthreads thread that could potentially access variables from a Perl interpreter running in another thread?
Liz
p5pRT
commented
6 years ago On Mon\, 09 Apr 2018 08:02:10 -0700\, davem wrote:
(I don't know what OOC stands for).
Probably âout of curiosityâ. Or maybe itâs a Dutch word.
--
Father Chrysostomos
p5pRT
commented
6 years ago On Mon\, 09 Apr 2018 12:37:54 +0200\, Dave Mitchell \davem@​iabyn\.com wrote:
Is this based on the idea that the ithreads implementation itself is inherently buggy\, or that some modules may not be thread-safe.
My current assumption is the latter. However i cannot speculate at all as to whether it might be some of column A or not.
2) If one's threaded code randomly craps out due to a 3rd party module being non-thread-safe\, or supposedly thread-safe but buggy\, then that's an issue with the third-party module. In principle any programming language will struggle with threads if used with a 3rd-party library that isn't thread-safe.
This has dangerous assumptions in it:
- Thread safety is not a binary on/off switch. Modules can appear to be thread-safe under some conditions and not be so under others. That's why i called it a heisenbug in my previous email.
- Putting the blame on 3rd party modules is fine for most languages where it's easier to switch to other implementations or roll your own. However this is Perl. The language lives on CPAN\, not in the interpreter. Asking someone who just spent hours implementing a parallel web downloader to gradually replace any of the bits they used will not be taken well.
- Very few cpan maintainers know how to do thread-safety. Heck\, most don't even practice string/binary safety for filehandles. Most of them are Perl developers first and you're asking them to move at least 2 degrees of separation outside of their zone of comfort. For a lot of modules there's simply a good chance this is not going to happen.
-- With regards\, Christian Walde
p5pRT
commented
6 years ago On Mon\, Apr 09\, 2018 at 07:26:30PM +0200\, Elizabeth Mattijsen wrote:
Ok\, so if I understand you correctly: if you run some pthreads outside of a Perl interpreter thread\, you will never be able to run any Perl code in that pthreads thread that could potentially access variables from a Perl interpreter running in another thread?
I'm not sure I understand that sentence\, but the basic rule of doing any sort of threading in perl is that two OS-level threads must not simultaneously access/modify the same perl interpreter struct (and all the data hanging off it\, e.g. SVs etc).
This was why the 5005THREADS model was abandoned. Nobody could think of a way of avoiding all the side-effects that any trivial usage of an SV might give rise to (e.g. changing the ref count\, converting from an IV to PVIV)\, without having a lock in about every second line of code in the perl core.
I vaguely understand that the approach in Python is to have one big central lock so only one thread can execute at any one time.
-- "I used to be with it\, but then they changed what âitâ was\, and now what Iâm with isnât it. And whatâs âitâ seems weird and scary to me." -- Grandpa Simpson (It will happen to you too.)
p5pRT
commented
6 years ago On Mon\, Apr 09\, 2018 at 11:01:22PM +0200\, Christian Walde wrote:
2) If one's threaded code randomly craps out due to a 3rd party module being non-thread-safe\, or supposedly thread-safe but buggy\, then that's an issue with the third-party module. In principle any programming language will struggle with threads if used with a 3rd-party library that isn't thread-safe.
This has dangerous assumptions in it:
- Thread safety is not a binary on/off switch. Modules can appear to be thread-safe under some conditions and not be so under others. That's why i called it a heisenbug in my previous email.
- Putting the blame on 3rd party modules is fine for most languages where it's easier to switch to other implementations or roll your own. However this is Perl. The language lives on CPAN\, not in the interpreter. Asking someone who just spent hours implementing a parallel web downloader to gradually replace any of the bits they used will not be taken well.
- Very few cpan maintainers know how to do thread-safety. Heck\, most don't even practice string/binary safety for filehandles. Most of them are Perl developers first and you're asking them to move at least 2 degrees of separation outside of their zone of comfort. For a lot of modules there's simply a good chance this is not going to happen.
But everything you've said above applies just as much to any 3rd-party library/module in any programming language which supports concurrency. I really fail to see how CPAN is different in this regard.
Most CPAN authors don't need to worry about thread-safety\, as long as it's a perl-level module. Perl is thread-safe at the language level\, and you don't need to do anything special to make Perl code thread-safe. Or at least\, nothing more than you would have to do to make your module safe across forks (which is what ithreads is emulating after all). For example\, if a thread is created\, the module's variables will be cloned\, and a destructor might be called for a variable in each of the threads. And of course if the module is updating something external like a file\, then it may need to exclusively lock that file.
XS code is of course a lot harder\, because now you're doing concurrency programming in C. But even here\, its easier than general C concurrency\, because of perl's ITHREADS "not shared by default" model. You mainly have to avoid global/static mutable data\, and be prepared for any of your data stored in SVs (e.g. DB handles) being cloned when a new thread is created.
-- All wight. I will give you one more chance. This time\, I want to hear no Wubens. No Weginalds. No Wudolf the wed-nosed weindeers. -- Life of Brian
p5pRT
commented
6 years ago On 10 April 2018 at 05:01\, Christian Walde \walde\.christian@​gmail\.com wrote:
On Mon\, 09 Apr 2018 12:37:54 +0200\, Dave Mitchell \davem@​iabyn\.com wrote:
Is this based on the idea that the ithreads implementation itself is
inherently buggy\, or that some modules may not be thread-safe.
My current assumption is the latter. However i cannot speculate at all as to whether it might be some of column A or not.
The official threads.pm documentation claims that the current code has what I'd class as "bugs":
Even with the latest version of Perl\, it is known that certain constructs with threads may result in warning messages concerning leaked scalars or unreferenced scalars. However\, such warnings are harmless\, and may safely be ignored.
If they are leaking\, this has bad implications for long-running code. If they're not leaking\, why the warning? If this information is out of date\, then the bugs are in the documentation!
2) If one's threaded code randomly craps out due to a 3rd party module
being non-thread-safe\, or supposedly thread-safe but buggy\, then that's an issue with the third-party module. In principle any programming language will struggle with threads if used with a 3rd-party library that isn't thread-safe.
This has dangerous assumptions in it:
- Thread safety is not a binary on/off switch. Modules can appear to be thread-safe under some conditions and not be so under others. That's why i called it a heisenbug in my previous email.
Indeed\, this is arguably worse than clear "the code crashed so it doesn't work" cases.
We also don't have good documentation on how to make modules thread-safe: I've encountered quite a few Perl developers who are confident that thread safety is only an issue with XS.
As an example: far as I recall\, the threads.pm documention and perlmod page to which they link never explicitly state that ref addresses will be different in each thread:
perl -e'use threads; use Scalar::Util qw(refaddr); my %x; $x{refaddr $_} = $_ for [qw(x)]\, [qw(y)]; threads->create(sub { warn threads->tid . " => " . join "\, "\, map { "$_ (" . refaddr($x{$_}) . ")" } sort keys %x })->join for 1..2' 1 => 30822192 (32355200)\, 30948440 (32355272) at -e line 1. 2 => 30822192 (32355456)\, 30948440 (32355528) at -e line 1.
I know much of my CPAN code will be susceptible to this type of issue. Any time someone asks about thread safety\, I point to the "discouraged" line in the official documentation and explain that I don't expect my code to work when multiple threads are active - "maybe try again if we have a new threads implementation in the future".
Downgrading that warning to something less clear could have unfortunate consequences for people learning (or supporting those who learn) Perl. I'm also not entirely convinced that "discouraged" is the wrong word - taking the phrases from the official policy:
From time to time\, we may mark language constructs and features which we
consider to have been mistakes as discouraged.
- Windows support: not a mistake. Exposing the API to end users without warnings and clear information about how threads differs from other languages\, on the other hand - maybe not ideal.
Discouraged features aren't currently candidates for removal\, but we may
later deprecate them if they're found to stand in the way of a significant improvement to the Perl core.
- surely this is the case? If we come up with a better way to implement threads while retaining the ability for Windows to support some form of process emulation and related features\, wouldn't this be something that's welcomed? I don't get the impression that there's widespread satisfaction with the current state of affairs on either threads.pm or threads::shared.
p5pRT
commented
6 years ago On Tue\, 10 Apr 2018 13:03:45 +0200\, Dave Mitchell \davem@​iabyn\.com wrote:
I really fail to see how CPAN is different in this regard.
Because this is Perl where not using CPAN is not an option and CPAN library consumers rely on CPAN libraries being almost entirely rock-solid\, or at least the author having easy and quick ways to fix bugs.
If you really look at all 3 points i made\, in aggregate\, and don't see how this is a problem and a danger\, then i don't think i can come up with other word combinations to make you see it.
-- With regards\, Christian Walde
Migrated from rt.perl.org#125106 (status was 'open')
Searchable as RT125106$