Move to pydantic v2

[DominicOram]

We should make sure that all our dependencies etc. work on pydantic v2

Acceptance Criteria

• We pin pydantic>2
• Unpin zocalo < 1.0.0
• We remove all the ignores around pydantic in filterwarnings
• Dependencies are happy to install on pip install
• Unit tests work

[DominicOram]

Currently blocked on:

• https://github.com/bluesky/scanspec/issues/122
• https://github.com/DiamondLightSource/python-zocalo/issues/247

[dperl-dls]

Once we can, we should then update some of the way we use pydantic features, e.g.: https://github.com/DiamondLightSource/dodal/blob/fe3f48cc7e65e05ca50025d22a2161bef4eadba3/src/dodal/devices/detector/detector.py#L59

[dperl-dls]

We are now blocked by scanspec requiring numpy>=2.0 and p4p requiring numpy<2.0

blueapi and nexgen also still pinned to pydantic<2 (so we can't pin to >2 and have a working environment)

additionally some changes are needed to get dodal in sync with ophyd_async 0.5+

• [x] https://github.com/DiamondLightSource/nexgen/issues/276
• [ ] https://github.com/epics-base/p4p/issues/145
• [ ] https://github.com/DiamondLightSource/blueapi/issues/273

[DominicOram]

Waiting for another release of p4p