Case for fully erased generics (syntax/reflection support only) for the time being

[autaut03]

Hey.

As there's been practically no movement whatsoever since 2019, I think it's fair to say that the approach of thinking every little detail through before actually adding the feature into PHP isn't working. This would have been a good plan if PHP had a much larger and/or active developer community, but as PHP is a nieche open-source product, I can see why there hasn't been any movement.

Current state of generics

Since 2019 some things have changed:

• IDEs started to support generics out-of-the-box. Namely, PHPStorm 2021.2, with much improved support in PHPStorm 2021.3
• static analysers have gained in popularity. According to JetBrains' development ecosystem surveys, 33% of asked do in fact use them. PHPStan used to be used by 9% of asked in 2019, it's now 18%
• PHPStan has released support for generics in 2019 and has been improving it ever since
• many projects, including major frameworks/libraries like Symfony, Laravel, Mockery & others already support generics in a form of PHPDoc

The community has already provided basic support for generics in their libraries & tools, but it will continue to be basic & limited without language-level syntax. Namely, the current PHPDoc approach has the following problems:

• inability to specify generic types while calling a function, including calling a constructor. Very basic things like typing DI's ->get(string $id) just aren't possible, as you can't specify type of the object you expect to be returned: ->get<MyService>('my_service_under_id')
• reflection is not available. Libraries have to use their own, very complex parsers & type systems. Psalm & PHPStan do it on their own, while other projects (like Rector) depend on them in a weird way to parse type information. It's also quite slow, it's not production-ready, it's not reliable and what's most important: there is no API. There are numerous use cases where such reflection API would be useful, namely: DI, serialization, bean validation, ORMs, annotation (attribute) based HTTP clients etc
• parsers don't have any knowledge of those annotations, hence some projects choose not to support them at all. php-cs-fixer is one of them and it's very understandable

Simpler version

These are the kind of problems that can not be solved without making changes to the language. Given the lack of resources to implement generics "in one go", I think it's worth focusing on releasing a simpler version.

A possible solution is the following:

1. Decide on generics design & syntax
2. Introduce parser-level support for generics, both in the language and https://github.com/nikic/PHP-Parser. No type or inheritance checking will be done at any point in time. Just the syntax.
3. Release

This will still require some effort, but not nearly as much as other approaches. I'd also like to note that I'm not proposing a bad, incomplete, less-feature rich or otherwise worse approach than any others. 35 active issues will still have to be solved, just not all at once. We need a basic working solution that isn't worse than PHPDoc, a base we can built upon later.

After that's done, IDEs, static analysers, other parsers & tools can start adopting or just supporting the new syntax. When they choose to drop support for old PHP versions, they'll be able to also drop their parser/reflection/annotations mess. Other things can be done afterwards:

• out-of-the-box reflection support
• adding type parameters to built-in types & functions
• reified types
• runtime and/or static type checking
• inheritance checking
• functional types (closures, callables)
• impact on/of strict_types, serialization, auto loading
• extended ::class

Drawbacks

There are obviously drawbacks to this approach:

• newcomers or people who haven't used generics will likely not understand why they aren't being runtime type-checked the same way other types are in PHP. While this is a problem worth solving, I don't believe those people are ones who need generics most. Those people can just choose to not use generics and it's perfectly valid
• without reification, things like instanceof T, var_dump(T::class) or new T won't work. However, not supporting those cases initially doesn't mean we can't add support for them later. This has worked fine for Java for many years, which also has runtime type checking (besides compile time, of course)
• we may need to introduce (expected) breaking changes as we go. For example, if we add inheritance checks, all existing code that uses generics might break. Although I think it's fair to assume there won't be many of those cases, as IDEs and static analysers will stop generics userbase from making mistakes, a attribute-based opt-in "experimental feature" mechanism could be introduced, similar to how Kotlin rolls out their features, allowing BC in later releases

Other than that, I think this is a more achievable goal than what we have in mind now.

[mindplay-dk]

Some objections:

1. As you pointed out, static type-checking tools are already available: this doesn't help anyone do anything new or different.

2. Purely static type-hinting without any run-time type-checking is unprecedented: this will be distracting and confusing for many users.

3. This will encourage users to write code that is apparently valid and runs - but at some point in the future might not be: things will break for reasons that likely won't be clear to many developers, who might not take any interest in a feature that doesn't bring any new functionality.

That said, yes, it does sound like the right place to start: it will enable some validation of syntax, concepts and ideas.

I'd like to suggest an alternative to actually modifying PHP: create a fork of PHP-Parser with support for generic syntax - this likely needs to happen anyhow for support in existing tools. Create an autoloader that simply removes generic type-hints - this will enable it to run, without needing to modify PHP itself.

I think this approach would enable the same kind of experimentation and validation of syntax and concepts?

[mrsuh]

Hi. I already has fork of PHP-parser with generics syntax support. Current parser has syntax limitations and we can't simply add all features we want. You can read about limitations here. And i already has library with "monomorphized generics autoloader". It support basic syntax and really works without needing to modify PHP itself. You can use it for your experiments.

[autaut03]

@mindplay-dk Thanks for the response.

1. I've also pointed out reasons why generics are severely lacking in functionality and use-cases without language-level support for them. This is something static analysers or the community will likely not solve on their own.
2. Agreed. This is one of the drawbacks. However, we have to consider that "many users" are actually people who already know what generics are and use them with PHPDoc. The set of users for this feature aren't all PHP users, but a tiny group of those who are already lacking runtime type-checking. It's also worth mentioning that we can always add runtime type-checking later.
3. Agreed.

I also agree that straight up adding it into PHP does not sound like an entirely good idea. There are several drawbacks and the API won't be stable.

Forking and adding support through an autoloader is a good idea. However, there are problems with this too:

1. stability of the autoloader. Is it stable enough to run in production, even given a high test coverage?
2. what about files loaded by libraries/frameworks with require?
3. add syntax support to PHPStorm - is this possible? It's not like it's an entirely new file type, so I'm not sure a plugin will be able to achieve this
4. how do we make tools' authors use the updated parser? I'd still want to run at least PHPStan and php-cs-fixer. I'd imagine PHPStan support can be added through a plugin, but that's definitely not the case with php-cs-fixer. There are also other tools out there, namely Psalm and php code sniffer, whose internals I know nothing about.

1-2 aren't that hard to overcome. 3-4 - I don't know if these are solvable. And without them, this implementation will remain an experiment. If that's what you meant, then sure, it's a good idea. But we'll need this implemented in PHP anyway.

@mrsuh Hey. Sounds good. If the point is to experiment, I'll be happy to.

[zmitic]

@mindplay-dk

Purely static type-hinting without any run-time type-checking is unprecedented: this will be distracting and confusing for many users.

True, it would be confusing for newcomers. But I doubt it will be that much of a problem in real use-cases if a big scary warning is put on php.net. Something like:

Generics are not using runtime type-checking, use them at your own responsibility. PHP team will not be held responsible for critical errors that might happen, dog eating your homework, or spouse leaving you for C# developer.

People wanting generics in PHP already use phpstan/psalm so I doubt we would be a problem.

---

There might also be an advantage of not having type checks, at least in the first version. My guess is that would make an easier transition and not introduce BC problems.

Use case: let's say there is /vendor code like this:

function doSomething(Collection $numbers): array
{
    return [42, 73];
}

Next version gets generics:

function doSomething(Collection<int> $numbers): array<int>
{
    return [42, 73];
}

I am no expert, but this looks like a potential for lots of problems if other code uses something like this. Think about complex dependency trees between different libraries where every single one would have to updated at once.

---

TL;DR:

Major/minor version for type-erased generics, with big scary warnings, to avoid BC problems.

Eventually, if runtime type-checking is added, new major version.

[autaut03]

Again, what about adding an opt-in mechanism for experimental features, allowing PHP to make breaking changes without fear?

Kotlin has such system. All of their new features are hidden behind @OptIn(FeatureName) annotation or a similar compiler flag. This notifies developers that the feature is not stable yet. Any additional nuances can be specified there too. Example: https://kotlinlang.org/docs/whatsnew14.html#converting-from-ktype-to-java-type

This means two things:

• generics would minimally affect users that don't use them yet. If they want to use the feature, they'll likely have to read the disclaimer and opt into it manually
• if we wanted to change the syntax, design, add runtime checks or drop generics completely, we'd be able to do so

Compared to trying to experiment outside of PHP:

• IDEs will add support for the new syntax, even if experimental, by themselves
• static analysers & other tools will do so too
• libraries & frameworks can start adopting the syntax much earlier
• no stability issues (in terms of being production-ready, not in terms of an API)

[mindplay-dk]

Hm, I don't know.

PHP developers expect run-time type-checks - this would be yet another inconsistency piled on top of an arguably already incoherent language with many legacy quirks. I think we'd all like to see PHP moving away from that, not making it worse?

In my opinion, this would be a poor feature without the run-time checks that PHP promises.

Even if the intention was to be a stop gap, there is a very real risk that a full implementation will never actually happen, leaving PHP with a limp leg for the rest of it's life.

(Also: YACC. Ugh. Isn't this always a roadblock for anything fun or fancy?)

[autaut03]

That's true. Developers will expect run-time type-checks. But it comes at a cost: for it to be runtime checked, PHP would have to solve all known and future issues at once before releasing even the first version. Realistically, this won't happen.

I understand that the proposed solution is very far from perfect. But it's either like this now and possibly improved later, or not supported at all any time soon.

So there are three roads:

1. we spend N more years waiting & likely doing nothing. Then M more years trying to solve all the oncoming issues and only then will we have the chance to release generics.

Outcome in N + M years:

• complete and stable feature
• possibly runtime checked, although not guaranteed. We might introduce a bundled static analyser that is ran automatically on local envs, who knows 🤷
• IDEs support the syntax
• starting to get adopted by applications, but not by libraries

Outcome in N + M + 1 years:

• libraries & frameworks have started adopting the "new" generics syntax
• most popular tools support the syntax
• new libraries for DI, serialization, validation etc have been released

Outcome in N + M + 2 years:

• libraries & frameworks have dropped support for generics-less PHP versions and released official generics support
• static analysers have dropped type-annotation parsing mess

2. we introduce fully erased generics into the language in the span of next year.

Outcome in 1 year:

• incomplete and experimental feature
• not checked out-of-the-box in any way
• IDEs support the syntax

Outcome in 2 years:

• libraries & frameworks have started adopting the "new" generics syntax
• most popular tools support the syntax
• new libraries for DI, serialization, validation etc have been released
• PHP prepares next major/minor version by this time which includes inheritance checks & keyword reified generics

Outcome in 3 years:

• libraries & frameworks have dropped support for generics-less PHP versions and released official generics support
• static analysers have dropped type-annotation parsing mess

3. we attempt to do it outside of official PHP.

Outcome in 1 year:

• incomplete and experimental feature
• not checked out-of-the-box in any way

Outcome in 2 years:

• possibly a third-party IDE plugin with syntax support
• possibly a third-party PHPStan plugin with full support
• other forked tools with syntax support
• starting to be adopted by applications, but not by libraries
• PHP returns to road # 1 or # 2, as there will be no further movement

Now, what do you think N and M are? Let's say we wait for 2 more years and then spend 2 more active years solving all the issues & preparing a major RFC/implementation. This means that best case scenario we'll get generics in 4 years.

Yes, they will likely be consistent with the rest of the type system. And yes, they'll be more or less feature-complete. At the same time people who already use generics, but want to majorly improve their user experience, can get them in just a year.

Even if the intention was to be a stop gap, there is a very real risk that a full implementation will never actually happen, leaving PHP with a limp leg for the rest of it's life. Also true. However, never implementing runtime checks for generics means one of two things:

1. PHP has dropped runtime checks and went for a different approach (e.g. a built-in static analyser of some sort)
2. PHP has stopped evolving, meaning it has become a dead language. In this case, I don't think it would matter if generics were fully implemented or not

Either outcome is fine.

[mindplay-dk]

I disagree.

What a project of this size actually needs is funding - it's really not fair or reasonable to expect developers to do this amount of very challenging work without pay, and it's probably not even something that ought to be attempted by one person without a team.

Just my opinion.

If you can get the 2/3 majority vote required to merge a half-feature into PHP, it's not for me to stop ya. 🙂

[autaut03]

Well, it’s also unreasonable to expect even a couple of developers to implement generics fully within a reasonable timeframe. If development begins now, generics are gonna be in PHP in 2-3 years the soonest.

What I’m saying is that it isn’t productive expecting miracles, especially in PHP’s case.

[zmitic]

@autaut03

Well, it’s also unreasonable to expect even a couple of developers to implement generics fully within a reasonable timeframe

To be honest, I think Nikita hacked his solution in a week or so, just for fun.

Could be wrong but I checked the tests and yes, it looks like they worked. Probably didn't put LSP checks but that is OK.

The idea was abandoned because it took too much memory; his words, we didn't get the numbers how big those are.

That was for reified generics. I am still convinced type-erased should be first to build to allow migration time, so 2-3 years does sound too much.

[autaut03]

@zmitic Type-erased won’t take nearly as much time because all major issues runtime checked generics have right now are simply avoided by design.

A prototype is good, but did it really account for everything? Memory usage, strict types, serialization, nested generics, ::type? Implementing just the monomorphized generics may be easy, but 35 others issues aren’t.

Trying to think through and implement most of the things will definitely not be quick. And that’s given there’s any movement to it, which there’s been none in the past years.

[Girgias]

I'm saying this as someone who has voting rights, but I will definitely vote against any proposal which is about just bringing erased generics.

The sheer addition of the syntax requires to move to a more complex parser makes this proposal already not worth it without all of the other problems already mentioned.

If you still feel that way and want to pursue an RFC, feel free to do so, but do not be surprised if many people on the internal mailing list will disagree that this is the best course of option.

[oprypkhantc]

@Girgias I understand. It'd be interesting to see what the mailing list would actually vote like, but I'm afraid you're right and most will be against the RFC.

And to be honest, I think part of the problem is that people who have voting power aren't representing modern PHP community. Almost 1/3 "no" votes on dynamic property deprecation RFC shows that :/

[Girgias]

@Girgias I understand. It'd be interesting to see what the mailing list would actually vote like, but I'm afraid you're right and most will be against the RFC.

And to be honest, I think part of the problem is that people who have voting power aren't representing modern PHP community. Almost 1/3 "no" votes on dynamic property deprecation RFC shows that :/

Ignoring that RFC because it ruffled some feathers, this has nothing to do about people in voting position not representing the modern PHP community. Many of the voters have voted for major BCs in 8.0 and 8.1. A disagreement about an extremely long standing behaviour has little to no impact on a new feature.

Moreover, "modern" PHP community doesn't mean anything, the PHP community is not limited to people who use a specific set of frameworks. Nor should, IMHO, internals care to some extent. Internals is there to design the language by improving it in a consistent manner while limiting BC breaks. This can mean deprecating edge cases or harmful features, but also adding features in a sensible manner.

And the addition of erased generic types is frankly not at all consistent with how PHP checks types, so breaking a fundamental assumption about the language is a bit no no for me. Just for the simple fact that if at a later point PHP starts actually type checking them, this would be considered a BC break, and that's assuming that the behaviour PHP decides on aligns with static analysers.

Just to say, bashing voters is frankly counter-productive and harmful, just because you disagree or don't know/understand a reasons doesn't allow you to make the claim that they are holding back the language.

[oprypkhantc]

@Girgias It is exactly about that. There were exactly two arguments:

1. "my or community's code has lots of use cases of dynamic properties", which translates to "I'm here to support legacy PHP and mostly ignore the rest"
2. "this is a big BC break", which was replied to many times: there were much bigger backward-incompatible changes in PHP 8 and yet they didn't raise quite as much attention

You're correct, internals is there to design the language. However, there are several paths that can be taken, and it's obviously up to internals to decide what's best. That doesn't mean it's best for everyone though. I don't think it's best for any of the PHP projects I'm working on, and I believe those projects represent a "modern" PHP community. Might be a bad choice of words, or I might be wrong about "modern" PHP community.

And again, you're being just as radical as @mindplay-dk. Unless they're perfect, you're against generics. You're completely right about expecting runtime checks, but you also have to account for reality. And the reality is that PHP doesn't have enough development power to implement runtime checked generics any time soon. So it might be against all fundamental assumptions, but it's either something or nothing at all.

I choose something. I don't see any reason why someone would want feature-less autocomplete-less ugly PHPDoc generics in favour of standartized language-level ones, so I'm making a reasonable assumption that everyone from a "modern" community would want something instead of nothing.

Regarding the BC break, I proposed a solution that's working well for a massive language that also promises BC.

What's also very counter-productive is ignoring real solutions and expecting miracles instead. Don't you think you also don't completely know/understand reasons for erased generics and hence think not having them doesn't hold back the language? Because so far, I haven't seen any other realistic better solution. So it's either you're waiting for more solutions (which are likely not coming) or don't understand the need for generics. Both of which are harmful for the language, imho.

[Girgias]

@Girgias It is exactly about that. There were exactly two arguments:

1. "my or community's code has lots of use cases of dynamic properties", which translates to "I'm here to support legacy PHP and mostly ignore the rest"

2. "this is a big BC break", which was replied to many times: there were much bigger backward-incompatible changes in PHP 8 and yet they didn't raise quite as much attention

You do know that I voted in favour of the deprecations because I found the argument about BC breaks weak? Moreover, many of the no votes came from people who usually do not vote. So again that specific RFC is really not the standard.

You're correct, internals is there to design the language. However, there are several paths that can be taken, and it's obviously up to internals to decide what's best. That doesn't mean it's best for everyone though. I don't think it's best for any of the PHP projects I'm working on, and I believe those projects represent a "modern" PHP community. Might be a bad choice of words, or I might be wrong about "modern" PHP community.

I don't know if you are talking about the dynamic props or not, but if you are, dynamic props are IMHO harmful, not now but also to future PHP devs. In regards to the community aspect, maybe "modern framework user", but there are people who write modern code without using frameworks, or who work on CMS plugins, etc. and to a certain extent we need to cater to all of them. And that's coming from someone who has strong opinions about various part within the ecosystem or the language itself (my first RFC was deprecating PHP's short tags).

And again, you're being just as radical as @mindplay-dk. Unless they're perfect, you're against generics. You're completely right about expecting runtime checks, but you also have to account for reality. And the reality is that PHP doesn't have enough development power to implement runtime checked generics any time soon. So it might be against all fundamental assumptions, but it's either something or nothing at all.

I choose something. I don't see any reason why someone would want feature-less autocomplete-less ugly PHPDoc generics in favour of standartized language-level ones, so I'm making a reasonable assumption that everyone from a "modern" community would want something instead of nothing.

That's rich to say I want something to be perfect when I never said this, especially as the person who implemented pure intersection types, a feature many do consider incomplete and far from perfect. I can see various ways of implementing a limited scope of generics while having them runtime checked, be that they are only usable on final classes/interfaces to bypass inheritance checks or otherwise. On top of that now that there is PHP foundation, having more core devs who can work on php-src at least part time is a real possibility.

Moreover, we lived years (even a decade) without scalar types, even less with proper union type support, even less with intersection type support, we don't even have DNF types yet (which I have a PR and RFC mostly ready to go), we do not have function types, etc. Waiting a couple more years to work on it properly instead (which if nobody beats me to it, I'm planning on spending some time on it at one point) of putting something in place which might make it impossible to actual have runtime checked types.

Regarding the BC break, I proposed a solution that's working well for a massive language that also promises BC.

Except if you're also autaut03 I don't know where you proposed your solution.

What's also very counter-productive is ignoring real solutions and expecting miracles instead. Don't you think you also don't completely know/understand reasons for erased generics and hence think not having them doesn't hold back the language? Because so far, I haven't seen any other realistic better solution. So it's either you're waiting for more solutions (which are likely not coming) or don't understand the need for generics. Both of which are harmful for the language, imho.

Myself, and others are not ignoring real solutions and expecting miracles, we just do not believe in rushing a solution as this has lead terrible decisions in the past (SPL data structures, magic quotes). As someone who's working on improving PHP's type system I (and others on the internals list) totally understand the need for generic types and how crucial they are to have a robust type system. However, this doesn't mean that we should rush and implemented something which we will need to carry with us if not forever, for a long time in case we messed up something.

Let me give an example of what I mean, which still raises question even if we limit generic types to only be usable on final classes. Lets have the following hierarchy of objects:

   A
 /  \
B1  B2
|
C

And let's have the typical use case of a Collection<T> class which has two methods push(T), pop(): T.

The easy case is that if the first element we add into the collection is of type A then any of B1, B2, and C should be pushable to the collection. But what if our first element we push to the collection turn out to be of type C, should T be bound to C? It could turn out we really wanted a Collection<B1> in this instance, so should this be rebound to B1 or even A as it is a direct parent and something of a very similar type to what was passed originally? And what if I pass an element of type B2 should this also be rebound then by checking if there is a common ancestor at a certain moment? Maybe instead of doing this guess work, the user must specify how to bind T. Something which is effectively an orthogonal feature to generics.

Frankly, I don't even know how any of the current PHP static analysers would interpret this as I haven't spent any time testing such weird cases with them.

Moreover, a static analysers can just decided those cases are too weird and throw an error, or change how it interprets it from one version to another, but if this becomes a language feature we need to think long and hard on which semantics we want as we're stuck with them, and this discussion needs to happen even if the proposal is only about erased generics at the time being.

Sadly due to PHP's current way of compiling files, we can't use techniques from compiled languages which have total knowledge of the usages and thus can make accurate type inferences. So maybe which should put more effort into PHP's compiler or a module/package system so that PHP can be aware of multiple files at once?

There are a magnitude of edge cases that need to be considered, and just saying "who cares we don't need them to be checked we just want nice syntax" to which I can reply

Just build a PHP preprocessor/transpiler that converts nice generic syntax into doc comments and run a static analyser on the output.

As one of my takes is that TypeScript is just that, a preprocessor and static analyser bundle together. Nothing is preventing the userland community from developing such a tool and embracing it while internals can work on integrating generics properly at the pace it needs. (side benefit such a preprocessor could have support for pure and/or immutable modifiers before they are, if ever, added to PHP.)

[natebrunette]

Maybe it would be best to not allow inferring the types based on first usage. That seems like it would add complexity for a little bit of syntactic sugar.

For example, you would have to specify

$collection = new Collection<B1>();

But could also do

$collection = new Collection<>([new B1()]);

Writing a userland pre-processor sounds OK in theory, but if industry-standard tools don't support the syntax, it's basically worthless, imo.

[oprypkhantc]

@Girgias

You do know that I voted in favour of the deprecations because I found the argument about BC breaks weak? Moreover, many of the no votes came from people who usually do not vote. So again that specific RFC is really not the standard.

You're taking this too personally. I never said anything about you specifically and never meant to. Whether those 1/3 of votes usually vote or not is irrelevant; what's important is that they can vote and they almost got the RFC to be rejected. Nothing is stopping them from doing so with other RFCs that break their 10 years old legacy wordpress plugin.

I don't know if you are talking about the dynamic props or not, but if you are, dynamic props are IMHO harmful, not now but also to future PHP devs. In regards to the community aspect, maybe "modern framework user", but there are people who write modern code without using frameworks, or who work on CMS plugins, etc. and to a certain extent we need to cater to all of them. And that's coming from someone who has strong opinions about various part within the ecosystem or the language itself (my first RFC was deprecating PHP's short tags).

No, I was talking about generics. I also don't get why you're mentioning frameworks for the second time here. I never mentioned any frameworks and I don't get how generics (or any RFC) are related to them. Any code can be modern, including CMS plugins and no-frameworks. If there's any correlation, I didn't mean it when mentioning "modern" community.

What I mean by "modern" is code that is kept up-to-date with evolving PHP & it's type system & ecosystem. Modern PHP versions and modern tools.

That's rich to say I want something to be perfect when I never said this, especially as the person who implemented pure intersection types, a feature many do consider incomplete and far from perfect.

That's fair. You never said that. What I meant is having runtime-checked generics requires solving most of the issues in this repository, to an extent we'd get pretty close to "perfect".

I can see various ways of implementing a limited scope of generics while having them runtime checked, be that they are only usable on final classes/interfaces to bypass inheritance checks or otherwise

Before merging even the first basic runtime-checked version, you'd still need to:

• implement reified generics (complex in of itself without monomorphization)
• solve serialization
• enforce variance
• strict_types impact
• ::type pseudo constant

Whether it's limited scope or not, it's still a lot of work.

Moreover, we lived years (even a decade) without scalar types, even less with proper union type support, even less with intersection type support, we don't even have DNF types yet (which I have a PR and RFC mostly ready to go), we do not have function types, etc.

And how was it? Pretty cool living without types, huh? It's like saying we lived years (or even hundreds of thousands) without computers and were completely fine! Isn't an argument.

Except if you're also autaut03 I don't know where you proposed your solution.

I'm also @autaut03, pardon me.

Myself, and others are not ignoring real solutions and expecting miracles, we just do not believe in rushing a solution as this has lead terrible decisions in the past (SPL data structures, magic quotes).

Well, again, we can experiment. C# does. Kotlin does. Pretty sure other languages do. Why can't PHP?

// Warning (or other error type) emitted here, saying GENERICS feature is experimental and doesn't provide BC.
function <T of object>test(): T {} 

// Same warning here
$var = test<stdClass>();

// No warning here
#[OptIn(Feature::GENERICS)]
$var = test<stdClass>();

// Require opt-in for definition of classes too
#[OptIn(Feature::GENERICS)]
class Test<in T, out K>{}

// File-wide opt-in
#file:[OptIn(Feature::GENERICS)]

This option requires a prior RFC for opt-ins, but this it's manageable and would benefit other future features as well. Or we can think of other ways of opting in (e.g. with declare()).

Regarding SPL, the solution in of itself (add standard data structures into the language) wasn't terrible. The implementation and lack of other features on top of them was. If PHP had extension methods, SPL structures were properly namespaced/named and were immutable, they would be priceless today.

So maybe instead of rejecting the decision so radically right away, reject specific implementations (of an RFC)? Again, with opt-in features, PHP can even reject them years after initial release.

Maybe instead of doing this guess work, the user must specify how to bind T. Something which is effectively an orthogonal feature to generics.

You don't have to have type inference. PHP can just as easily force users to specify types all the time. Java used to not have any inference on variable types whatsoever. You always had to specify the type. They lived like this for 10+ years and were mostly fine. Again, it's better to have to specify types manually than not having generics at all.

In your cases, that would be Collection<A> for both cases, unless you want to limit the collection to C. In that case, you would get an error the second you try to push A/B1/B2 into the collection.

Frankly, I don't even know how any of the current PHP static analysers would interpret this as I haven't spent any time testing such weird cases with them.

Well, they just don't infer types when they can't. Simple as that. You specify the type manually if it can't be inferred from the constructor.

Moreover, a static analysers can just decided those cases are too weird and throw an error, or change how it interprets it from one version to another, but if this becomes a language feature we need to think long and hard on which semantics we want as we're stuck with them, and this discussion needs to happen even if the proposal is only about erased generics at the time being.

And why so? It obviously needs to happen if we want runtime-checked generics, but with type-erased ones PHP wouldn't care. Static analysers will do the job (well, they already did) and PHP can just adopt a ready solution afterwards.

Or if you really want to have a single standard on how static analysers should interpret/handle generics, I don't think that's a big problem. We already have whole two static analysers which have had generics support for years now. They both agreed on semantics and both offer an identical syntax & feature set.

PHP can borrow whatever they need from them, as it's what's been working for real PHP projects for the past two years. It's also pretty much the same as what we have in other languages (just a lot less feature-complete), for which it's also been working great.

Sadly due to PHP's current way of compiling files, we can't use techniques from compiled languages which have total knowledge of the usages and thus can make accurate type inferences. So maybe which should put more effort into PHP's compiler or a module/package system so that PHP can be aware of multiple files at once?

I'm aware of the limitation. But is it necessary for generics? Not really. A nice syntax sugar - sure, but a necessity - definitely not.

There are a magnitude of edge cases that need to be considered, and just saying "who cares we don't need them to be checked we just want nice syntax" to which I can reply

Sure. And there are 10x the amount of edge cases for any kind of runtime checked generics.

As one of my takes is that TypeScript is just that, a preprocessor and static analyser bundle together. Nothing is preventing the userland community from developing such a tool and embracing it while internals can work on integrating generics properly at the pace it needs. (side benefit such a preprocessor could have support for pure and/or immutable modifiers before they are, if ever, added to PHP.)

Yes. And TypeScript is a tremendous effort. Spec, official IDE support, backward compatibility with JS (.d.ts included), preprocessor, countless tests. It's been 9 years and to this day many libraries are not just written JS, they don't even have first party TS support.

Technically possible - yes. Realistic to expect libraries, frameworks, IDEs, tools to support it, and the community to actually implement a production-ready stable version any time soon? Don't think so. Frankly, this would be even more effort than runtime checked generics.

[mindplay-dk]

And again, you're being just as radical as @mindplay-dk. Unless they're perfect, you're against generics.

It's kind of ridiculous to accuse me of being against generics. I wrote the RFC.

I also don't expect perfection. I think it's perfectly fine to start with a subset of a feature palette.

What I disagree with, is the idea of starting with a horizontal slice (a half-feature) rather than a vertical slice (a sub-feature) - I've already explained why, so I won't do that again.

You're completely right about expecting runtime checks, but you also have to account for reality. And the reality is that PHP doesn't have enough development power to implement runtime checked generics any time soon. So it might be against all fundamental assumptions, but it's either something or nothing at all.

I choose something. I don't see any reason why someone would want feature-less autocomplete-less ugly PHPDoc generics in favour of standartized language-level ones, so I'm making a reasonable assumption that everyone from a "modern" community would want something instead of nothing.

If that's the process, I am fine with that too.

But a half-feature belongs in fork, not in a mainstream release - for those who are interested and willing to experiment with a half-feature, installing a fork is perfectly normal and fine. For those who rely on PHP for their livelihood, not having access to a feature that may or may not get finished, is probably safer and a better use of their time.

Node.js for one have always had two releases - a stable release and one with experimental features.

I don't think that's radical, I think that's just how software is normally built.

[mindplay-dk]

(and if you're somehow able to construe generics without type-checking as as vertical slice, in a language that consistently type-checks all type-hints, I disagree - it's a departure in an entirely different direction. It's comparable with TypeScript to JavaScript - it serves a different set of requirements and a different workflow. Expectations for someone working with TS is very different from someone working with JS. PHP, to my knowledge, doesn't have any compile-time only features. In my opinion, that's a much more radical departure than it might seem like.)

[autaut03]

It's kind of ridiculous to accuse me of being against generics. I wrote the RFC. I also don't expect perfection. I think it's perfectly fine to start with a subset of a feature palette.

I've never said anyone was against generics in general. You're against non runtime-checked generics, which given the amount of issues that need to be solved to support that, practically means they would be near-perfect.

for those who are interested and willing to experiment with a half-feature, installing a fork is perfectly normal and fine

What do you mean exactly by "willing to experiment"? Are you talking about development-only pet projects or real-world live projects?

If it's the former, sure. But that's not the intent. You and I both proposed a much better solution than a fork: experimental features. But somehow I haven't got replied to after mentioning it three times already.

I think that's just how software is normally built.

It might be true for PHP. Other software - not really. Most software offers alpha & beta versions, which do not promise BC. Speaking of programming languages, look at Kotlin, C#, JS, TS, Rust. They all offer some form of experimental features.

[mindplay-dk]

look at Kotlin, C#, JS, TS, Rust. They all offer some form of experimental features.

In their mainstream (non-beta) releases? (Examples, please.) 🙂

[mindplay-dk]

Most software offers alpha & beta versions, which do not promise BC.

Just to be clear, I'm not opposed to that either.

Part of it is just how they label their releases - PHP calls them beta releases, while e.g. Node describes them as separate mainlines, but eventually everything from their "Latest Features" series moves up and replaces the "LTS" series, which is "recommended for most users", in part because their "Latest Features" release series sometimes contains breaking changes. So they describe it differently, but the "Latest Features" release is really just a (often more long-lived) series of "beta" releases.

So by all means, let's have experimental, half-baked generics features in betas.

My only point is, these should not come out of beta as half-features - this stuff shouldn't get mainlined to developers with the risk of having them build with language features that don't fully work, or might break.

That's what "stable" means, as far as my understanding. It doesn't just mean "it runs without breaking down" or something - for a programming language, especially, it should mean "we think these features are here to stay."

[autaut03]

@mindplay-dk

My only point is, these should not come out of beta as half-features - this stuff shouldn't get mainlined to developers with the risk of having them build with language features that don't fully work, or might break.

Well, the point is to not mark the whole release as "experimental", but to mark specific features as experimental in an otherwise stable release (or rather just a release, not necessarily stable one. PHP doesn't have a stable version per say). And to make sure developers know things can break or change, we can force them to acknowledge that through the use of compile flags, runtime flags, attributes, declares etc.

In their mainstream (non-beta) releases? (Examples, please.) 🙂

Yes. Although I admit I lied regarding Rust (unintentionally). Some examples:

1. kotlin:

   • new api - string extensions: https://kotlinlang.org/docs/whatsnew1430.html#locale-agnostic-api-for-upper-lowercasing-text
   • new syntax - inline classes: https://kotlinlang.org/docs/whatsnew13.html#inline-classes
   • new types - unsigned integers: https://kotlinlang.org/docs/whatsnew13.html#unsigned-integers
   • whole new package - serialization, bundled with Kotlin compiler: https://kotlinlang.org/docs/whatsnew13.html#kotlinx-serialization
   • coroutines: https://kotlinlang.org/docs/whatsnew11.html#coroutines-experimental
   • others & stability explanation: https://kotlinlang.org/docs/components-stability.html

   All of these require either a compilation flag or an @OptIn(FeatureName) annotation. IDE & the compiler will not let you use the feature unless you opt-in.

2. typescript:

   • new syntax & api - decorators: https://www.typescriptlang.org/docs/handbook/decorators.html

   Again, IDE & compiler force you to opt-in.

3. c#:

   • new syntax - nullable reference types: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/nullable-reference-types

   Same once again. Forced to opt-in.

[mindplay-dk]

So Kotlin actually mainlines experimental feature and red flags them in the documentation. Interesting. That's one way of doing it, I suppose.

As for your C# example, this is not an experimental feature - the flags are just project-wide compiler options. Maybe your point with this example was to show how the feature got enhanced at a later time? I would view the introduction of this feature as a new feature - the introduction of the compiler option doesn't break anything existing or change existing behavior, so the addition is horizontal, as I see it.

I'm not aware of C# having experimental features on the mainline. I did some searching and found this - you can set the compiler version explicitly, and you can set it to preview to get experimental features that haven't been versioned yet. So they do have experimental features, but they're not enabled without an opt-in.

TypeScript is similar, putting optional features behind a flag - although here, the opt-ins are for individual features. I expect that's much harder to do in practice, even in a compiler that was built with a modular design to enable opt-ins. That particular compiler option has been there since the dawn of time, and I think it might actually be the only one - so maybe this isn't even easy in compilers that were designed for it.

The PHP parser almost definitely wasn't designed for it - and how would you opt-in anyway, there's no such thing as a configuration-file for PHP projects.

So two of your three examples don't really help your case. But I don't know, maybe red flagging features in documentation like Kotlin does is fine.

Either way, as mentioned, it's not me you have to convince, but the panel of product owners who get to vote on feature inclusions. If you want to know what they think, you'd have to go on the PHP internals mailing list and ask them.

I'd be curious to hear what they think. 🙂

[autaut03]

and how would you opt-in anyway, there's no such thing as a configuration-file for PHP projects.

Well, attributes (which could be made available file-wide, like in Kotlin) or an existing PHP mechanism - declare().

So Kotlin actually mainlines experimental feature and red flags them in the documentation. Interesting. That's one way of doing it, I suppose. So two of your three examples don't really help your case. But I don't know, maybe red flagging features in documentation like Kotlin does is fine.

Not only in the docs; the compiler actually requires you to use either the -opt-in compilation flag or the @OptIn(Feature.SomeFeature) annotation. The flag is obvious; the annotation needs to be present whenever an experimental feature is used. So it does help my use case:

// when this code is ran (or compiled, depending on what's easier/better), it throws a notice (or a different PHP error type)
class Collection<TValue, TKey of int|string = int> implements IteratorAggregate<TKey, TValue> {}

// all good here
#[OptIn(Feature::GENERICS)]
class Collection<TValue, TKey of int|string = int> implements IteratorAggregate<TKey, TValue> {}

// again, thrown here
$items = new Collection<int>();

// good here
#[OptIn(Feature::GENERICS)]
$items = new Collection<int>();

#file:[OptIn(Feature::GENERICS)]
// file wide like declare

I'd imagine this wouldn't be the easiest thing to implement, so maybe sticking with (file-wide) declares instead is a better idea:

declare(generics=1);

class Collection<TValue, TKey of int|string = int> implements IteratorAggregate<TKey, TValue> {}
$items = new Collection<int>();

Either way, as mentioned, it's not me you have to convince, but the panel of product owners who get to vote on feature inclusions. If you want to know what they think, you'd have to go on the PHP internals mailing list and ask them.

I'll try that. Although having an easier communication method would help :)

[mindplay-dk]

File-wide declares, how would that work?

What if you enable generics in one file, and I import it from a file where generics is not enabled?

Generics seems like it has to be a language/project-wide opt-in. Likely the reason Kotlin can do that, is either because the effects of these flags are localized, and/or because the language is statically-compiled - it's going to see your annotations at compile-time, no matter which file it's in, and the compiler can configure accordingly.

(This makes me think again about something I've mentioned in the past. If you were to start from the idea of "types as values" (fully "reified") what would that look like? Your type parameters would literally just be instance "properties" containing "values" of the type "type". That's how it works in Dart, another language where types and type-hints are generally fully reified - that's much closer in nature to PHP than generics are in Java or TypeScript, and lends itself better to things like reflection. Start with something that is closer to PHP. Worry about static constraints, validations, optimizations and stuff later on - while that's a big part of the value of generics, PHP generally doesn't have much in the way of static validations for anything more than syntax. You rely on tools for that stuff anyway.)

[damek24]

Hi, As I'm PHP developer i'd love to see this feature (even incomplete) IMO there are a few options to add generics feature (however I don't know how hard it would be to implement them):

1. as @autaut03 mentioned @OptIn attribute
2. generics as php extension (disabled by default)
3. compilation flag (like with-generics / with-experimental)
4. integrated into PHP core and configurable via php.ini (sth like generics-enabled / experimental-enabled) and disabled by default

or maybe at first (let's say PHP8.2) do not implement generics at all and just focus on implementing something like this (but built in language itself) https://github.com/dmitrya2e/php-generic-collection, where a type is defined by the annotation ?

[razshare]

Being concerned about new developers writing code that might be not valid code in the future is a good idea, ofc, but then why not start implementing the syntax behind an experimental runtime flag?

Not to focus too much on java but they have been doing this for some time and it's been working great. I don't see people complaining about the java fibers api changing every release, people know it's experimental.

A well named flag that tells you "you're on your own" should be fine no?

[someniatko]

@tncrazvan Splitting the language in two is not a good idea:

• some libraries may embrace the new feature and thus render clients with the flag turned off incompatible (however this could be handled similar to an extension)
• introduction of generics is way too invasive both for PHP internals and for the visible behavior of the language. It means PHP development team will have to support two not so slightly different languages at the same time until the flag is gone. However, PHP team is not a large one to handle such a burden.

[autaut03]

@someniatko PHP is free to choose the implementation of experimental flags. It could just as well work on per-file basis, the same way strict_types work.

So addressing your first concern, a library using an experimental feature may not require it's users to also use the experimental feature. I don't see why we would want such a requirement.

Regarding the language, well, it's a single language, with a single syntax. It's not like it would be split into different branches or released separately. Just a bigger codebase, with one more major feature. I see why it's a burden for PHP, but at the same time PHP is getting close to a point where there aren't many more important improvements to be made, so maybe focusing on generics for a year or two is a good idea.

[olleharstedt]

No one did an RFC on this? On the php.net wiki, that is.

[autaut03]

@olleharstedt That'd be great, but this was just a discussion of a general idea. No details have been agreed upon yet, although a lot has been discussed in other issues in the repo.

Also, I don't believe there's even a slightest chance of generics being accepted into PHP in this incomplete form without first discussing how they'll evolve and, what's more important, how breaking changes are to be introduced in the future. There's currently no mechanism in PHP that would allow breaking changes to existing syntax/APIs (what's called experimental features in other languages) and I believe that's a must for an idea like this.

It's not to say that breaking changes will have to be made, but without this fail-safe I wouldn't expect it to gain 2/3 votes. And even with a fail-safe I'm still not convinced it would pass, especially coming from someone entirely outside of php-src contributors. That's why I personally didn't go ahead with an RFC.

[olleharstedt]

how breaking changes are to be introduced in the future

Hm what kind of breaking changes are you thinking of? What I see in this thread, no changes needed to add generics annotations would break old code.

[oprypkhantc]

Hm what kind of breaking changes are you thinking of? What I see in this thread, no changes needed to add generics annotations would break old code.

I mean after releasing the fully erased generics. We might need to change the syntax in edge cases, or add type checks if we go with reified generics. Both would be a massive concern for php-internals and will have to be thought of ahead-of-time.

[olleharstedt]

Oh sure, going from erased generics to type-checked generics must not change syntax, I agree with that. Do you know any edge-cases that are complicated in this regard?

[oprypkhantc]

Oh sure, going from erased generics to type-checked generics must not change syntax, I agree with that. Do you know any edge-cases that are complicated in this regard?

Type-checked generics are in of itself a breaking change, because existing (fully valid and running) code might break after adding type checks if generic types were incorrect in the first place. After all, using type-erased generics doesn't guarantee also using a static analysis tool, meaning mistakes are unavoidable.

Not aware of any edge-cases, but I'm sure there are plenty of. I'd be afraid of them if I was part of the internals 🙃

[mindplay-dk]

Type-checked generics are in of itself a breaking change, because existing (fully valid and running) code might break after adding type checks if generic types were incorrect in the first place. After all, using type-erased generics doesn't guarantee also using a static analysis tool, meaning mistakes are unavoidable.

This is one reason I'd prefer to start with a smaller, but more complete set of features - adding type-hints with no run-time type-checking is unprecedented in PHP, which by now is a progressively-typed language. I don't imagine internals would vote yes to include a feature that isn't "functional" in the PHP sense of types being checked and reflected - it's how the PHP type system works.

Of course, nothing prevents anyone from starting and maintaining a fork, until the full feature is ready for RFC. But I can imagine where such a fork would end up though. I don't have the C skills to build this, but if I did, I'd try to reduce the scope. I understand that's what you're doing as well, but rather than carving out a vertical slice, I'd try to narrow the horizontal - it's what I'd do with any software development task if it's too big. Build fewer things, but build them to completion - makes more sense to me than building more but incomplete things.

I've never actually worked on a language though, so maybe the situation is different from other software. 😊

[autaut03]

I've never actually worked on a language though

Me neither. I'm just not sure there's anything to cut on the horizontal scale. We could cut off function generics and even type variance, but I don't have the competence to say whether that'll actually change the implementation complexity by a lot because these problems sound like they have relatively simple solutions. And the part we can't cut off (class-like generics with inheritance support and reification for runtime checks) sounds like the hardest part to implement with no clear solutions as suggested by nikic's attempts :(