Open jcrist opened 2 years ago
So I'm obviously somewhat biased here but I do want to share some thoughts on the pros:
Meta.__iter__
be implemented. I think this should be pretty straightforward to do and it can even be done lazily (e.g. importing the annotated-types
types within __iter__
so that it is zero-cost if this functionality is not used).pydantic.BaseModel
by msgspec.Struct
, for the most part). I think given the recent emphasis on Pydantic feature parity this would be very interesting. This would require that msgspec be able to understand annotated-type
which might require moving some parsing of the types from C to Python (or a lot of work in C, which you seem to handle no problem š ).In my personal projects I have been using a fork with iter implemented (via crude hack but seems to work alright)
I have seen pyparsing used to parse a constraint string minilang into actual annotated types as in AndreaCensi/PyContracts. I would imagine implementing a stack-based language is beyond the scope of msgspec but I wanted to mention it because I had fiddled with the idea of the minilang as an actual C extension but found it a bit reinventing the wheel-y
I am happy with the current system, if it works with static typing. If it doesn't, I would love for it to either be updated to do so, or if the performance hit weren't large, to support annotated-types. Here is my test case, I may be doing something wrong (constraint documentation specifies that the current system is supported by mypy):
from typing import Annotated
from msgspec import Struct, Meta
UnitFloatValidator = Annotated[
float, Meta(ge=0.0, le=1.0)
]
Length = Annotated[list[int], Meta(max_length=3)]
class ProbabilityInterval(Struct):
"""Represent an interval of probabilities."""
lower_bound: UnitFloatValidator
upper_bound: UnitFloatValidator
class Length3String(Struct):
""" A string of length 3"""
some_string: Length
if __name__ == "__main__":
# These don't fail, but should
print(ProbabilityInterval(lower_bound=-1, upper_bound=1.1))
print(Length3String(some_string=[1,2,3,4]))
# Arguably this is ok not to fail, because Python ints and floats are compatible
print(ProbabilityInterval(lower_bound=int(1), upper_bound=1.0))
# These fail
print(Length3String(some_string=["a", "b", "c"]))
print(ProbabilityInterval(lower_bound=[1], upper_bound=1.1))
The cases that don't fail are missed by both mypy and pyright, in my hands at least.
Grateful for all the work you've done on this btw, and your willingness to engage the community.
To my knowledge no static analysis tool (mypy
, pyright
, or otherwise) currently has support for doing anything with annotations added via an Annotated[...]
(annotated-types
or otherwise). One of the arguments in the PEP proposing typing.Annotated
was that type checkers could ignore these extra annotations and have them still be available at runtime.
To support these kind of constructs in mypy
you'd need to write a custom mypy plugin. AFAIK there's no way to add a similar plugin to pyright
.
Ah I misunderstood annotated-types.
Iām going to give this solution a try: https://stackoverflow.com/questions/32787411/how-to-declare-python-constraint-on-generic-type-to-support-lt
Hi @jcrist a followup discussion here: https://peps.python.org/pep-0746/ and https://discuss.python.org/t/pep-746-typedmetadata-for-type-checking-of-pep-593-annotated/53834.
Even if msgspec doesn't use annotated-types I'd still really appreciate your input on that PEP since msgspec can adopt it even without annotated-types. In particular on the signature of the method (type vs. value). Thanks!
came looking to see whether there had been discussion of using annotated-types here, and found this. Just wanted to put in my support for msgspec supporting annotated-types annotations. Wouldn't mean depending on it, just basically "honoring" it such that Annotated[int, annotated_types.Ge(0)]
had the same semantics as Annotated[int, msgspec.Meta(ge=0)]
, etc...
I'm someone who definitely likes to go back and forth between msgspec and pydantic, and also generally likes to support concepts that are re-usable and shareable. annotated-types is a nice initiative for merely marking the concept of a constraint, in a simple-to-depend-on package, while imposing almost no assumptions about implementation details. š
PEP 746 certainly looks nice as well :)
Follow up from #176 and #154.
We currently support adding constraints to types by annotating them with
msgspec.Meta
.annotated-types
provides it's own annotations describing the same constraints - we could potentially integrate with them in 2 ways:Meta
annotations.Meta.__iter__
produce the equivalentannotated-types
annotations, letting it work with other tools that consume such types.I'm not convinced there's a real use case for either of these integrations, and would want to hear from multiple real users before expending any effort supporting this. See https://github.com/jcrist/msgspec/pull/176#issuecomment-1229265714 for a longer write up of my thoughts.