Lack of native threading?

[lanforce]

Description

Why this is still not a part of the core? Why can I just import threading in Python and use it right there without any hackinations?

PHP is like a white crow in the field of concurrency & computational capabilities, but yeah, we're focusing on some lazy objects to clean up Symfony... Sure the most wanted feature (not), lol.

[cmb69]

Why this is still not a part of the core?

The author had plans to suggest to integrate it into the Zend engine, but had been discouraged due to fibers having made it first. Maybe it is something we should consider. Perhaps @realFlowControl has some thoughts on this.

Anyhow, please stay civil here at least. Thanks!

[lanforce]

@cmb69 Wow, you're like a breath of fresh air here. I expected this issue to be closed almost immediately. 🤣

Will try to be civilized. Seeing all other languages have good support for concurrency with almost 0 configuration and then a lot of PHP folks tend to just "don't care" made me cry. And that's because I love PHP and wish it to be a better place in the parallelism/concurrency/computational field. It's already the king in the business area I would say, like this article highligths.

If PHP fixed those lackings, and maybe added a native performant production-ready HTTP server, this would neglect what other languages offer so we could stay in PHP for most of the use cases.

[NattyNarwhal]

I think historically, concurrency/parallelism for incoming requests tended to be orchestrated outside the PHP application (i.e. it came free with your web server starting PHP scripts), and most people weren't doing heavy MT computation in PHP. Fibres came in as Christoph mentioned, and take care a lot of remaining I/O situations you'd want threads for.

[determin1st]

im against threading. it is unsafe and the speed benefit vs process is not obvious. i dont really know what those NTS and TS mean for now, but ive already deleted fibers because they are not needed for asynchronicity (as well as generator)

added a native performant production-ready HTTP server

yes, as an extension without any additional branding, like a Curl extension is about Curl.

I think historically, concurrency/parallelism for incoming requests tended to be orchestrated outside the PHP application (i.e. it came free with your web server starting PHP scripts)

it is still the case. even the built-in web server php has (i deleted it). look at it as a closed resource, closed from the php file and the php code itself. php doesnt really have a grasp of those servers, they dont extend php. again, if given the interface like Curl's, php could abstract and wrap it into a nice asynchronous API

[cmb69]

[…], and most people weren't doing heavy MT computation in PHP.

What might be a chicken-and-egg problem. To my knowledge, there is only one publicly available extension for multi-threading, which may not be well-known, and for scientific computing there is Tensor which may be even less known, and I suppose is still rather suboptimal on Windows (due to the supporting libraries not being optimized there).

im against threading. it is unsafe […].

I think that mostly depends on the API. Low-level APIs (like these as the old pthreads) are hard to use correctly, but it seems to me that nowadays there are pretty easy to use (and such safe) APIs.

Actually, it's quite funny to see myself arguing in favor of multi-threading (I loved to switch to PHP for not having to deal with such issues), but nowadays some applications can't do without.

[lanforce]

im against threading. it is unsafe and the speed benefit vs process is not obvious. i dont really know what those NTS and TS mean for now, but ive already deleted fibers because they are not needed for asynchronicity (as well as generator)

Threads are a tool. Such statement could only sound from the person who never used threads in cases where they're an appropriate solution for the problem. Don't be a Shrek in your own swamp, we should have tools other languages have for decades. I've used threads from PHP's parallel extension and it's GOLD for my use cases. But somehow I need to find a maintained ZTS build and do a lot of unnecessary configuration in PHP to make it work smoothly, while in Python it works out-of-the-box with 0 dependencies and 0 configuration. Why PHP should somehow be worse than Python?

[bwoebi]

@lanforce you'll always need a thread safe build of PHP to do anything multithreading in PHP. How hard is it for you to use parallel? Like you're saying "a lot of unnecessary configuration", when it should be just downloading and extension=parallel.so

[pierrejoye]

On Wed, Sep 4, 2024, 11:19 PM determin1st @.***> wrote:

im against threading. it is unsafe and the speed benefit vs process is not obvious. i dont really know what those NTS and TS mean for now, but ive already deleted fibers because they are not needed for asynchronicity

TS enables locks and related ops when needed, where nts does nothing, while sharing the same macros/functions.

as of thread vs not threaded, you may find frankenphp interesting, or refreshing.

[lanforce]

@lanforce you'll always need a thread-safe build of PHP to do anything multithreading in PHP. How hard is it for you to use parallel? Like you're saying "a lot of unnecessary configuration", when it should be just downloading and extension=parallel.so

Dude, have you ever used PHP threads in production? It sounds like you were toying with it and nothing more. Imagine how it's for DevOps to maintain two separate versions of PHP (one NTS, second ZTS) in the cloud, putting different configurations for web requests and console modes to achieve the best performance and separation of process management. You don't do any downloading and extension=parallel.so for this in Python, and you don't maintain two language versions. I hope you at least understand why I don't use the ZTS build for serving the web requests.

P.S. Thanks to people like you, we broadly hear that PHP doesn't have threading and it's not a language to use if you want something more than the web.

[cmb69]

I don't see the need to discuss configuration options here. I rather see this as a feature request to have some userland MT capabilities always "built-in" (for ZTS builds, of course). In the past we've made ext/json and ext/hash mandatory, just so every user can rely on the extension being available (regardless of configuration, maybe even by shared hosting). And fibers have been adapted via https://wiki.php.net/rfc/fibers, which states:

Adding this capability directly in PHP core makes it widely available on any host providing PHP. Often users are not able to determine what extensions may be available in a particular hosting environment, are unsure of how to install extensions, or do not want to install 3rd-party extensions. With fibers in PHP core, any library author may use the feature without concerns for portability.

It sounds like you were toying with it and nothing more.

I don't think such assumptions are helpful.

[pierrejoye]

On Thu, Sep 5, 2024, 11:14 PM lanforce @.***> wrote:

@lanforce https://github.com/lanforce you'll always need a thread-safe build of PHP to do anything multithreading in PHP. How hard is it for you to use parallel? Like you're saying "a lot of unnecessary configuration", when it should be just downloading and extension=parallel.so

Dude, have you ever used PHP threads in production? It sounds like you were toying with it and nothing more. this because you commented.

I reiterate my suggestion about frankenphp (or other). Without code change.

also, as a side important note, please reconsider your tone and how you address other people here. Thank you :)

[determin1st]

@lanforce

Threads are a tool. Such statement could only sound from the person who never used threads in cases where they're an appropriate solution for the problem. Don't be a Shrek in your own swamp, we should have tools other languages have for decades.

yes, low-level tool. PHP is bigger than that. for example, when i do asynchronous Curl with my wrapper, i know that Curl creates threads under the hat. but i dont need to create threads for PHP code. so im gonna stay big big elephant :]

I've used threads from PHP's parallel extension and it's GOLD for my use cases.

i saw some PHP thread code examples, dont think process variant will be much much worse. im projecting it now, most of complexity lies in synchronization/communication. so describe your usecases, maybe they dont need threads

But somehow I need to find a maintained ZTS build and do a lot of unnecessary configuration in PHP to make it work smoothly, while in Python it works out-of-the-box with 0 dependencies and 0 configuration. Why PHP should somehow be worse than Python?

threading implies more dependencies, i draw a simple scheme to demonstrate that. also think about the benefit of separation, so the PHP process to process runtimes may do only intended work, while threaded versions must include everything

Adding this capability directly in PHP core makes it widely available on any host providing PHP. Often users are not able to determine what extensions may be available in a particular hosting environment, are unsure of how to install extensions, or do not want to install 3rd-party extensions. With fibers in PHP core, any library author may use the feature without concerns for portability.

i doubt that fibers can serve any need without any further abstractions in a form of a library or what people doing these days - huge huge frameworks. i saw its code does blocking/unblocking in a different zend places so it is probably a requirement to have it (fibers) bound with the core, not because it can be useful for everybody. yes, we know that laravel is popular, but it doesnt mean everybody use laravel, i dont use laravel

[lanforce]

threading implies more dependencies, i draw a simple scheme to demonstrate that.

Again, why Python don't have any problems with that? You guys being so mindblocked by those "demonstrations" won't do anything helpful to the language users.

[lanforce]

I reiterate my suggestion about frankenphp (or other). Without code change.

You offer me to shift to third-party Golang solution? Thanks. Maybe I should consider learning Golang, not PHP after all?

[lanforce]

I don't think such assumptions are helpful.

also, as a side important note, please reconsider your tone and how you address other people here.

I don't think you guys focus on what's important.

[krakjoe]

This isn't a feature most people want. We can count on our fingers the number of serious conversations we've had about adding threading to userland in the last 20 years.

It's easy to assume your requirements are common, it may feel extremely important to you but in the wider scheme of things it just isn't.

It's interesting enough for me to have spent time on it and at one point planned a userland API. The project went another way when we chose fibers, in my opinion that was the end of the road for a core API; If we merged parallel now we would be adding only one feature, and footguns too numerous to list here. That would not be a net win.

That's all I have to say about that ...

Cheers Joe

On Sat, 7 Sept 2024, 07:34 lanforce, @.***> wrote:

I don't think such assumptions are helpful.

also, as a side important note, please reconsider your tone and how you address other people here.

I don't think you guys focus on what's important.

— Reply to this email directly, view it on GitHub https://github.com/php/php-src/issues/15726#issuecomment-2335051232, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARB52WQ4WPSOVOADW6KGQDZVKF6JAVCNFSM6AAAAABNSRRNI2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZVGA2TCMRTGI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[lanforce]

It's easy to assume your requirements are common, it may feel extremely important to you but in the wider scheme of things it just isn't.

You know, people look at the language tooling when making decisions. Knowing PHP doesn't have a native async/await, doesn't have threads and other low-level stuff other languages have out of the box, they decide not to go with PHP. They think they'll reach the point when it'll be needed, and then it'll require the adoption of another tool that has it, for example, Python. Then why not go with Python from the start? I've seen that multiple times in my career and it's blown my mind. PHP is considered not a general-purpose programming language, but a strictly web-oriented scripting language. And seeing PHP core people arguing we don't need threads makes me think these decision-makers are right, at least to some extent.

[pierrejoye]

On Sat, Sep 7, 2024, 12:30 PM lanforce @.***> wrote:

I reiterate my suggestion about frankenphp (or other). Without code change.

You offer me to shift to third-party Golang solution? Thanks. Maybe I should consider learning Golang, not PHP after all?

the reply was specifically about TS vs NTS. ts is used by frankenphp sapi.

[cmb69]

the reply was specifically about TS vs NTS. ts is used by frankenphp sapi.

Right, but this is not what this feature request is about. It is about some possibility to run computationally expensive userland code in PHP using multiple threads (without installing an external extension); and that is likely not something users want to do in a Web SAPI.

The project went another way when we chose fibers, in my opinion that was the end of the road for a core API; If we merged parallel now we would be adding only one feature, and footguns too numerous to list here. That would not be a net win.

Since you are way more knowlegdable about multi threading than me, I take your word for it. Still, I feel that PHP should not completely ignore the need for multi-threading nowadays. I don't think that anybody would use e.g. Python for computationally expensive code, if they had no native support for multi-threading; Numba and NumPy appear to be later additions.

[determin1st]

Knowing PHP doesn't have a native async/await, doesn't have threads and other low-level stuff other languages have out of the box, they decide not to go with PHP

nah, ive decided to code in PHP because it didnt have Promises/A and let/consts integrated (imo var is better). "native" means design is locked, not a better design. for example JS Promises dont define a cancellation, also, they are active (like fibers) not passive, they start "ticking" right after you create an object, similar to how garbage collector operates. await is not special, it can be a function, like await(giveMeSomePromise()), it doesnt have to be an echo("something") operator

They think they'll reach the point when it'll be needed, and then it'll require the adoption of another tool that has it,

ive reached the reverse point with fibers injected, sadly, :] for me, but can fix those for my runtime. no code produces no bugs.

...for example, Python. Then why not go with Python from the start? Again, why Python don't have any problems with that? You guys being so mindblocked by those "demonstrations" won't do anything helpful to the language users.

python has lots and lots of problems with concurrency (like it cant do await in await as NODEJS does). i recognize that python is better at CLI (traditionally). im not mindblocked, i asked your threaded examples, you escaped :]

PHP is considered not a general-purpose programming language, but a strictly web-oriented scripting language.

CLI will fix that. there is a readline extension which i think should be PHP code, not binary and it should be based on terminal abstraction that is also should be PHP code, not compiled. python has those.

And seeing PHP core people arguing we don't need threads makes me think these decision-makers are right, at least to some extent.

i see no argument for thread model. i think switching threads or processes doesnt matter for performance of PHP code (otherwise why you need it?). showcase that asynchronicity can go below milliseconds (OS switching thread/process is less than a millisecond). here, for example https://www.man7.org/linux/man-pages/man2/nanosleep.2.html in HISTORY it says it cant go below 2ms for any userland thing. windows SleepEx cant go below milliseconds. maybe threads are more convenient? (show your samples)

i said "unsafe" before because i dont understand what will happen if fatal error is triggered in the threaded PHP code, like what will happen:

# in some PHP thread
throw new \Exception('test');

when this is not catched? ive made a test with my process to process communication and know how to handle that, master checks the process is not running, reads its stdout pipe and extracts those remnants of fatal message. with threads, i assume the best scenario is that all gracefully terminates and message is delivered to the common stdout

[lanforce]

im not mindblocked, i asked your threaded examples, you escaped :]

I mostly use it for stateless computation which doesn't require any interprocess communication. The possibility of having it in a "lightweight" thread instead of an entire process saves a lot of memory for me. You could say this is possible with something like queues. Yes, kind of. But not quite. Queues are for a different purpose. And still, this would require full-fledged processes and the need to maintain the connection pool... Why should I build a complex sidecar infrastructure for PHP for something that could be possible on the language level without any dependencies?

[determin1st]

I mostly use it for stateless computation which doesn't require any interprocess communication.

communication always happens. you give computator data/options and get back the result, thats a communication.

The possibility of having it in a "lightweight" thread instead of an entire process saves a lot of memory for me.

lightweight thread needs heavyweight process. imo, lightweight process is better. even 2 processes :] i launched about 1800 processes before getting memory limit message

You could say this is possible with something like queues. Yes, kind of. But not quite. Queues are for a different purpose.

not quite. i project it the following way. theres a global class Process, first it has to initialize (subscribe to process idea):

Process::init([/* options */]);

after that, we're in business, slave (computator) can receive a "command" from the master with the data through the handler function it supplied in options or with Process::set_handler(/* function */) do the work required and report back with Process::event(/* data */) can report the progress as well

And still, this would require full-fledged processes and the need to maintain the connection pool...

full-fledged sound big, but sucha simple PHP process is about 12Mb (in memory consumption) on my machine, probably can be smaller if removed other features. theres no "connections", i use Sync extension for IPC through shared memory. have to keep track on PIDs to be able to send them commands and distinguish events. but its the same with threads, isnt it?

Why should I build a complex sidecar infrastructure for PHP for something that could be possible on the language level without any dependencies?

either we compete with sidecar implementations or there will be one dependency for everybody. ima proponent of competition. my implementation is better than fibers, ive tested http client of amp and phasync that utilize fibers they arent faster than my curl wrapper and promises but their dependency tree is tens of megabytes. why you think that there is a magician that will make "official" implementation better? official doesnt always mean better

[pierrejoye]

On Sat, Sep 7, 2024, 7:27 PM Christoph M. Becker @.***> wrote:

the reply was specifically about TS vs NTS. ts is used by frankenphp sapi.

Right, but this is not what this feature request is about. It is about some possibility to run computationally expensive userland code in PHP using multiple threads (without installing an external extension); and that is likely not something users want to do in a Web SAPI.

he stated TS is not stable nor usable for anything, many amazing new sapis made this statement invalid.

btw, that's my reply, the rest is another person's reply ;-)

The project went another way when we chose fibers, in my opinion that was

the end of the road for a core API; If we merged parallel now we would be adding only one feature, and footguns too numerous to list here. That would not be a net win.

Since you are way more knowlegdable about multi threading than me, I take your word for it. Still, I feel that PHP should not completely ignore the need for multi-threading nowadays.

i have the same feeling. However it requires to rethink a tat bit what TS does, and do it more efficiently.

A solid ts model is a pre requise for parallelism.

Go, Dart have good userlands (language constructs and interfaces) f.e. However it still requires some good clue about what to do and when to avoid race conditions or other multi threading issues on cross threads (or whatever is used) shared data.

That being said, there are also different needs:

• parallelism
• concurrency

Parallelism is completely absent in core but as the request sapi level, more or less. More would be needed to be effective, like vector data etc.

concurrency is there since long. Fiber makes it easier, whether everyone likes it or not is a different story :)

[realFlowControl]

Hey @lanforce

Seeing all other languages have good support for concurrency with almost 0 configuration [...]

But somehow I need to find a maintained ZTS build and do a lot of unnecessary configuration in PHP to make it work smoothly.

I do not see the one or the other being a problem. The (in)official docker containers have ZTS (Zend Thread Safe, or TS) tags. Also I do not know which configuration you are referring to, ext-parallel does not have any configuration.

So it is:

FROM php:8.3-zts
RUN pecl install parallel \
 && docker-php-ext-enable parallel

I do see that not really many folks are aware of that fact, that you actually can do user-land threads thanks to ext-parallel, and that this might mean that some people do not use PHP in case they need / want to solve their problems using multithreading because of that. Also I do see that the TS vs. NTS thing is something folks from other ecosystems are not aware of, maybe the parallel docs could be a bit better about this.

In case you see anything specific (in regards to TS vs NTS or configuration that you were referring to), please feel free to reach out to me via an issue or pull-request at https://github.com/krakjoe/parallel and help making it more easy to discover or setup.

And hey there @determin1st 👋

i said "unsafe" before because i dont understand what will happen if fatal error is triggered in the threaded PHP code, like what will happen:

# in some PHP thread
throw new \Exception('test');

when this is not catched? ive made a test with my process to process communication and know how to handle that, master checks the process is not running, reads its stdout pipe and extracts those remnants of fatal message. with threads, i assume the best scenario is that all gracefully terminates and message is delivered to the common stdout.

The \parallel\run() function or \parallel\Runtime::run() function will return a \parallel\Future that represents the return value or unhandled exception, see this example:

$future = \parallel\run(function() {
    return 2/0;
});

try {
    $val = $future->value();
    var_dump($val);
} catch (\Throwable $e) {
    var_dump($e);
}

[determin1st]

@realFlowControl

The \parallel\run() function or \parallel\Runtime::run() function will return a \parallel\Future that represents the return value or unhandled exception, see this example:

$future = \parallel\run(function() {
    return 2/0;
});

try {
    $val = $future->value();
    var_dump($val);
} catch (\Throwable $e) {
    var_dump($e);
}

yes, i see, it is no different to synchronous code. even with this example i dont understand how it works (where it starts running?):

$future = \parallel\run(function() {
    return 2/0;
});
\sleep(3);# CAN IT THROW HERE?
try {
    $val = $future->value();
    var_dump($val);
} catch (\Throwable $e) {
    var_dump($e);
}
# CAN IT THROW HERE?

i suppose the value() blocks until thread exits so the second Q uestion falls off, then, if the first is negative, it means the main thread will be always in a blocked condition while the spawned thread is running - theres no point in such a parallelism. so i assume otherwise.

what will happen with other threads, will they be informed of the master shutdown (due to unhandled exception) or they will be promptly terminated? im asking basics of parallelism - how it starts and how it stops. i think those are key moments one should encounter first in documentation. it is not obvious or self-evident, and, it is not like with synchronous operation

looked at some recent Swoole documentation (https://wiki.swoole.com/en/#/thread/thread), it also added threads, so at least there are 2 variants to choose from. let author decide which variant he wants injected into the core

i can propose the following example suitable for comparison:

• both master and slave (or main thread and spawned thread, or parent-child // naming depends on parallelism supplied terminology) do something, lets call it a "work".
• one finishes faster than the other, reports it to the console and if it is a master, it awaits the slave, the slave is allowed to exit without waiting for the master.

this will showcase the parallelism and a bit of concurrency used by supplier. readability and comprehension of the code will be a criteria for user selection.

the "work" can be a simple random sleep, like 4-8 seconds span, maybe with some echo so it will be clearly seen that threads intertwine. what you think