pydantic 2 - migration - Githubissues

experimental-design / bofire

Experimental design and (multi-objective) bayesian optimization.

https://experimental-design.github.io/bofire/

BSD 3-Clause "New" or "Revised" License

222 stars 23 forks source link

pydantic 2 - migration #279

Closed bertiqwerty closed 9 months ago

bertiqwerty commented 1 year ago

The goal of this MR is to upgrade to Pydantic 2. For the initial commit I run bump-pydantic and changed some things manually. There are still some open issues, though.

jduerholt commented 10 months ago

Todo list:

review all model validators, we should switch everywhere to mode "after", this is more failsafe
add validators for inputs and outputs objects to check for unique keys
intotroduce commone type for unique str tuples

jduerholt commented 10 months ago

There are some bugs in pydantic2:

Order of AnyFeature affects the correct deserialization
cannot deal with update_hyperparams as abstract method ...

Things to consider:

Look at order_id for features
make one place for custom types

jduerholt commented 10 months ago

@bertiqwerty: Only failing tests are now due to failure in the DOE module since on the 25th of December a new formulaic version was released and this breaks some of our tests.

jduerholt commented 10 months ago

Hi @bertiqwerty,

tests are now passing, and I think the PR is ready for review.

What do you think, who should review it?

Issues that are mentioned above which are still open are:

create a central place for our custom types like bofire/data_models/types.py
on several places in the code we use lists that constrain strings which have to be unique (examples are the features attribute from constraints, the categories attribute from categorical features etc.). In my opinion we should switch them all to tuples and create a custom type for this.

But it could be better to do this in a seperate PR.

What do you think?

Furthermore, we need to think about a merging/release strategy as this PR introduces quite a big change/break.

Best,

Johannes

bertiqwerty commented 9 months ago

Hi @jduerholt ,

Thanks a lot! The person for review would probably be Benny. I will also have a look. Having a central types.py makes sense to me. We should do this in a separate PR.

jduerholt commented 9 months ago

Hi @bertiqwerty,

thx! Yes, I will go through it and then approve and merge. For me it works.

The idea with the reportUnnecessaryTypeIgnoreComment is a good one! Maybe we should also switch to a more recent pyright? We are still on version 1.1.305.

Before than merging it in, should we make a last pydantic v1 release?

Best,

Johannes

bertiqwerty commented 9 months ago

Hi Johannes,

I think we don't need another release with Pydantic 1 but I also don't have an objection. So I am fine either way.

Updating Pyright is a good idea. This often leads to some issues. So we could create a MR for this. I can look into this.

jduerholt commented 9 months ago

Ok, then we do not create a Pydantic 1 release. Would be nice if you can take care for the pyright update. But we should do it after merging in this PR. My plan is to have it in the latest tomorrow. Note that I moved the pyright config some time ago into the pyproject.toml, so the new flag for showing the unnecessary ignore statements should also go there, I think:

https://github.com/experimental-design/bofire/blob/33a2053e80c371791cb27364c442b83f866b1c09/pyproject.toml#L34

jduerholt commented 9 months ago

@bertiqwerty: it is now merged, you can now have a look at the pyright update ;)

bertiqwerty commented 9 months ago

I am in favor of making a new release with the new Pydantic version.

jduerholt commented 9 months ago

Ok, fine for me. Do you want to do it or should I do it?

bertiqwerty commented 9 months ago

I can.