JakobGM / patito

A data modelling layer built on top of polars and pydantic
MIT License
270 stars 23 forks source link

[FEAT] Upgrade to `Pydantic v2` #28

Closed jjfantini closed 6 months ago

jjfantini commented 10 months ago

This will allow multiple benefits.

Use cases:

I need to validate a df that can have multiple names for a specific column. Let's say it could be col1 or `col[1-10]'.

I have a df that is created, and the column names can be a few different things but the same column, and I need a way to validate all the possible names. I've tried multiple things, custom validators, creating a model dynamically, and they have yet to work.

jjfantini commented 10 months ago

@JakobGM

brendancooley commented 10 months ago

Running list of todos here, perhaps we can farm these off into narrower issues if the implied solution is precise enough:


some wip on validators here. The data frame validation logic doesn't quite map to the way pydantic v2 expects validation to occur, so I more or less ported over the pydantic v1 validation error reporting system into patito. It looks like this is where @thomasaarholt was headed with his PR. all tests in test_validators now passing except those that have serialization issues with json_schema_extra fields.


wip on extending FieldInfo here


fully operational prototype for the upgrade is up at #32