CarliJoy / intersection_examples

Python Typing Intersection examples
MIT License
33 stars 2 forks source link

How to proceed from here #45

Closed mikeshardmind closed 8 months ago

mikeshardmind commented 8 months ago

Maybe this is a "me issue" and not that this can't be done right now, but I'm unsure how to meaningfully make progress in specifying OrderedIntersection. As far as I'm concerned, it should be as simple as.

For something to be consistent with assignment to a variable with a type specification of OrderedIntersection, it must be consistent with all operands of the OrderedIntersection. For the use of something that is an OrderedIntersection to be consisdered consistent, it must be consistent with the first relevant type found in the OrderedIntersection.

All of the rationale for that is already covered over in #38 , the rationale and explaining the rationale is not the issue I'm having.

Based on a lot of other discussions about the typing spec, I don't think it should be specified much more than what is above.

Examples of practical effects and comparisons, and why that's the decision? sure, but this should be extremely clear that these are examples based on the current state for the PEP process, and the examples should not be taken as limitation on intersections, and instead should be updated if the examples ever become wrong due to other changes to other types.

Based on the direction of other parts of typing, it doesn't seem like such a sparse definition will be accepted. I think overspecifying this is a road to compositional incompatabilities.

For instance, the discussion we had about this with TypedDicts. If we decided to say any more and actually say it can't compose TypedDicts with non-typed dicts only because there is no legal way to do this currently, then the spec for intersection has precluded a change to TypedDict that could allow it. The consistency based definition will allow a user to specify such an intersection, but they won't be able to create something to assign to that intersection (without a cast). Such uses then allow disovering what the use cases for expanding on TypedDict may be, and only the specification for typed dict would need to change for the use to be fully legal later on.

The consistency-based rule is rather simple and leaves what is and isn't compatible to the types within the intersection, but the entire direction of the typing spec seems to be trying to come up with increasingly precise, narrow, and even intentionally-limited definitions instead of definitions that state the minimum behavior and generalize, and It's been hard to find a way to describe intersections that doesn't feel wrong as a result, either for being out of place in the specification and unlikely to be accepted by those who want precise and narrow definitions, or for being precise and narrow when that's more limiting to all future constructs without needing to be.

I could use some other guidance here because if I'm off base, and I should just write the simpler definition and explicitly reject a more narrow one in the PEP itself to avoid the issues of composition and that is a direction we want, then I can finish the draft much sooner as there I need far less language to go this route, and I think I've been expending too much time trying to reconcile narrow definitions.

mark-todd commented 8 months ago

Once again thanks for working on developing the PEP for this. I think for a PEP to be accepted it will probably need to describe how it deals with the edge cases we've been discussing, but that doesn't mean we have to do it all in one go. If you have simple definition that is described well, that seems like a good starting point to me, and we can expand on it with edge cases and things that we think will provoke questions like TypedDicts.

We could even have a point of "FAQs" - things that we think aren't necessary to define it but will be useful to people trying to think about these edge cases.

I would suggest when you're happy with a basic structure for it (and perhaps some headings for the edge cases like typed dicts) we can iterate on it a bit. We could open PRs to suggest additional paragraphs/examples and discuss wording.

mikeshardmind commented 8 months ago

need to describe how it deals with the edge cases [...] we can expand on it with edge cases and things that we think will provoke questions like TypedDicts.

This part in particular worries me. As far as I'm concerned, there are no edge cases to the definition above. The definition does not require a single special case or knowledge of any existing or future type feature to remain consistent long term, but we do want to provide examples that demonstrate this in the PEP, without those examples becoming binding on the specification. OrderedIntersection[T, U] should automatically as a consequence of the definitions and even specification of T or U changing "just work" with the updated definitions. Many of the examples that exist in the specification have been used to argue intent, and in many of those cases arguing that was even appropriate, but in this case, the intent is that the examples are only that way because they would be reflective of the behavior, not that the existing behavior that comes from T or U that we'd be describing a resolution of is in any way key to OrderedIntersection.

It's a very intentionally abstract feature because it describes an interaction between other concepts, and I'm unsure of how to write this in a way that will be accepted in such an abstract manner, to the point where I've spent a couple weeks (well, free time within those days) first attempting not to, surrounding context of that being the recent typing discussions on discourse (not discord) and the direction of them.

DiscordLiz commented 8 months ago

I've seen the over-specialization you mentioned, so I don't want to be telling you you're overthinking this, but you're definitely letting your other experiences with the specification process get in the way of writing what you intended. You managed to take something people couldn't agree on, and come up with an abstract definition that simultaneously wasn't quite an intersection anymore, but did resolve all of the issues people couldn't find agreement on.

You already made it clear you're not going to be happy presenting something you think makes the situation worse, so present what you intended and don't compromise on the presentation of it, but be ready to defend or clarify it as needed.

mark-todd commented 8 months ago

This part in particular worries me. As far as I'm concerned, there are no edge cases to the definition above.

Just of point of clarification here, sorry I realise now I was using the term "edge case" slightly loosely. I didn't mean cases that fall outside of the specified definition, but more cases that will be points of contention or questions. These should already be defined under the abstract definition, but just for clarity I'm saying we spell out what the abstract definition implies under conditions that might lead to discourse.

It's a very intentionally abstract feature because it describes an interaction between other concepts, and I'm unsure of how to write this in a way that will be accepted in such an abstract manner.

The definition can be abstract so long as the examples of how the definition applies in practice are descriptive. I think as long as we describe the applications of the definition I see no reason why it wouldn't be accepted.

mark-todd commented 8 months ago

For something to be consistent with assignment to a variable with a type specification of OrderedIntersection, it must be consistent with all operands of the OrderedIntersection. For the use of something that is an OrderedIntersection to be considered consistent, it must be consistent with the first relevant type found in the OrderedIntersection.

Also just a point on this - I think this definition is correct, but there are a couple of points here we should clarify. There's actually quite a lot of behaviour built into the terms "consistent" and "relevant", so we should explain what each of those mean in this context. E.g. "consistent" might include something about how Protocols don't require an inheritance but other classes do - or what it means for a TypedDict to be consistent structurally. (I'm not saying my definition of consistent is complete, just trying to illustrate a more descriptive definition of terms can be used to explain why it covers many cases)

mikeshardmind commented 8 months ago

Not fully ready for a review by others here, but I'm linking the draft that I need to turn into a PR or something later. If anyone has any comments on what's there, feel free, but I'll be turning it into something more reviewable once I handle formatting, the todos, and choosing which examples make the case the best.

https://gist.github.com/mikeshardmind/990a05a9673ca18238affa19dcc5cd8b