Support custom validator functions

[alexei]

Description

I think it would be awesome to be able to set callbacks as constraints:

Country = Annotated[str, msgspec.Meta(callback=lambda x: x in [...list of country codes...])]

Birthday = Annotated[date, msgspec.Meta(callback=lambda x: x <= date.today()]

etc.

Also note https://github.com/jcrist/msgspec/issues/482

[jcrist]

Thanks for opening this, this is something I'd like to support.

I think we'd want to spell things a bit different to allow for custom error messages - if the callback returns false it's not clear what info the error message should contain. Instead I think we'd do something like:

def not_in_future(x: datetime.date):
    if x >= datetime.date.today():
        raise ValueError("date must be <= today")
    return x

Birthday = Annotated[datetime.date, msgspec.Meta(validator=not_in_future)]

[jcrist]

Also, note that your first example would currently be better expressed using a Literal or Enum type.

[alexei]

I like the idea of raising ValueError 👍

[aedify-swi]

@jcrist Can I still assist with specifying the use cases or is this clear to you? This would be super useful to us. We're currently using __post_init__, but these validators would really help because we could skip them when initialising the class directly (needed for some tests). I hope this doesn't sound too pushy, just want to make sure this is on your radar.

[ml31415]

Would be great to have that. This would also solve https://github.com/jcrist/msgspec/issues/425

[alexei]

Seeing as the ticket is still open, I'll take the opportunity to share a few thoughts that I've been having recently. When I opened this, I think I was enthusiastic to play with constraints and didn't give it much thought. While I was busy asking myself how to do validation with msgspec, I did not ask myself whether msgspec should do any validation at all. A recent experience has made me reconsider this.

Between a single library that did serialisation, validation and conversion poorly, and three libraries that each excelled at one of those things, I think I'd prefer the latter. This isn't a critique at msgspec. It's a reminder that extra features and extended functionality come with their own costs and challenges. So far I found myself using msgspec for serialisation alone. To be honest, I've been pretty satisfied with that. The simplicity is what caught my eye about this library in the first place. The world doesn't need another pydantic.

[gsakkis]

Between a single library that did serialisation, validation and conversion poorly, and three libraries that each excelled at one of those things, I think I'd prefer the latter.

So which two libraries that excel at validation and conversion respectively do you use?

Btw, that's literally the first sentence of the README:

msgspec is a fast serialization and validation library,

[ml31415]

@alexei I don't see your point. Sounds a bit like the linux command line philosophy, except it just doesn't make sense to apply it here. If you need validation without dealing with JSON, just use pydantic. If you don't need the validation, just parse your objects with msgspec into plain dictionaries, without giving any struct. Otherwise, the validation - especially without adding much of a time penalty - is one of the unique selling points of this library. And if validation is already defined as being an integral part of this library, also adding customization options like presented in this issue make perfect sense.

[alexei]

@gsakkis @ml31415 thanks for your input

[nathanielarking]

Heyo, this would be an instant reason for me to switch to msgspec from pydantic, just wondering what the status is on something like this being implemented?