Pydantic v2 migration issue

joostlek commented 10 months ago

The problem

Pydantic released v2 of their library. Some libraries want to utilize the new features of Pydantic v2 so HA should update to Pydantic v2. To not burden all libraries to rework, Pydantic is providing v1 in a separate package of v2. This has been implemented in several libraries already made by some HA community members (https://github.com/AngellusMortis/pyunifiprotect/pull/297). This way the library supports both v1 and v2, this way of working is preferred to not burden the v2 upgrade with a lot of libraries to update at once.

After installing all dependencies and using pipdeptree we see that the current libraries use pydantic.

pydantic==1.10.12
├── aiolivisi==0.0.19 [requires: pydantic]
├── aionotion==2023.5.5 [requires: pydantic>=1.10.7,<2.0.0]
├── aioopenexchangerates==0.4.0 [requires: pydantic>=1.9,<2.0]
├── aiopurpleair==2022.12.1 [requires: pydantic>=1.10.2,<2.0.0]
├── aiowaqi==0.2.1 [requires: pydantic>=1.10.8]
├── demetriek==0.4.0 [requires: pydantic>=1.9.0,<2.0.0]
├── elgato==4.0.1 [requires: pydantic>=1.8.0,<2.0.0]
├── gcal-sync==4.1.4 [requires: pydantic>=1.9.0,<2.0a]
├── google-nest-sdm==2.2.5 [requires: pydantic>=1.10.4]
├── ical==5.0.1 [requires: pydantic>=1.9.1]
│   ├── gcal-sync==4.1.4 [requires: ical>=4.2.5]
│   └── pyrainbird==4.0.0 [requires: ical>=4.2.9]
├── inflect==6.0.4 [requires: pydantic>=1.9.1]
│   ├── jaraco.itertools==6.2.1 [requires: inflect]
│   │   └── jaraco.abode==3.3.0 [requires: jaraco.itertools]
│   └── jaraco.text==3.11.1 [requires: inflect]
│       ├── jaraco.collections==4.2.0 [requires: jaraco.text]
│       │   ├── jaraco.abode==3.3.0 [requires: jaraco.collections]
│       │   ├── jaraco.email==3.1.0 [requires: jaraco.collections]
│       │   │   └── jaraco.net==9.3.1 [requires: jaraco.email]
│       │   │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       │   └── jaraco.net==9.3.1 [requires: jaraco.collections]
│       │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       ├── jaraco.email==3.1.0 [requires: jaraco.text>=1.3]
│       │   └── jaraco.net==9.3.1 [requires: jaraco.email]
│       │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       └── jaraco.net==9.3.1 [requires: jaraco.text]
│           └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
├── intellifire4py==2.2.2 [requires: pydantic]
├── lacrosse-view==1.0.1 [requires: pydantic>=1.9.0]
├── open-meteo==0.2.1 [requires: pydantic>=1.8.0,<2.0.0]
├── peco==0.0.29 [requires: pydantic>=1.9.0]
├── pvo==1.0.0 [requires: pydantic>=1.8.0,<2.0.0]
├── pyaussiebb==0.0.15 [requires: pydantic>=1.9.0,<2.0.0]
├── pyrainbird==4.0.0 [requires: pydantic>=1.10.4]
├── python-bsblan==0.5.11 [requires: pydantic>=1.9.0]
├── python-kasa==0.5.3 [requires: pydantic>=1,<2]
├── python-opensky==0.2.0 [requires: pydantic>=1.10.8]
├── pytraccar==1.0.0 [requires: pydantic>=1,<2]
├── pyunifiprotect==4.10.6 [requires: pydantic!=1.9.1]
├── radios==0.1.1 [requires: pydantic>=1.9,<2.0]
├── sfrbox-api==0.0.6 [requires: pydantic>=1.10.2]
├── systembridgeconnector==3.4.9 [requires: pydantic>=1.9.0]
├── tailscale==0.2.0 [requires: pydantic>=1.8.0,<2.0.0]
├── vehicle==1.0.1 [requires: pydantic>=1.8.0,<2.0.0]
├── withings-api==2.4.0 [requires: pydantic>=1.7.2,<2.0.0]
├── xbox-webapi==2.0.11 [requires: pydantic]
├── yolink-api==0.3.0 [requires: pydantic>=1.9.0]
├── youtubeaio==1.1.5 [requires: pydantic>=1.10.8]
└── zwave-js-server-python==0.50.1 [requires: pydantic>=1.10.0]

This issue will be the progress tracker of this migration.

❌ aiolivisi

Library: [aiolivisi](https://github.com/StefanIacobLivisi/aiolivisi) Integrations using this library: [Livisi](https://github.com/home-assistant/core/tree/dev/homeassistant/components/livisi) Code owners: @StefanIacobLivisi @planbnet - [ ] Supports Pydantic v2 (PR: https://github.com/StefanIacobLivisi/aiolivisi/pull/12) - [ ] Is updated in HA (PR: # ) Remarks:

❌ aionotion

Library: [aionotion](https://github.com/bachya/aionotion) Integrations using this library: [Notion](https://github.com/home-assistant/core/tree/dev/homeassistant/components/notion) Code owners: @bachya - [x] Supports Pydantic v2 (PR: https://github.com/bachya/aionotion/pull/251) - [ ] Is updated in HA (PR: # ) Remarks: There seems to be no version that supports both v1 and v2, so this library should be directly bumped with the v2 update.

❌ aioopenexchangerates

-- Library: [aioopenexchangerates](https://github.com/MartinHjelmare/aioopenexchangerates) Integrations using this library: [OpenExchangeRates](https://github.com/home-assistant/core/tree/dev/homeassistant/components/openexchangerates) Code owners: @MartinHjelmare - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

❌ aiopurpleair

-- Library: [aiopurpleair](https://github.com/bachya/aiopurpleair) Integrations using this library: [PurpleAir](https://github.com/home-assistant/core/tree/dev/homeassistant/components/purpleair) Code owners: @bachya - [x] Supports Pydantic v2 (PR: https://github.com/bachya/aiopurpleair/pull/195) - [ ] Is updated in HA (PR: # ) Remarks: There seems to be no version that supports both v1 and v2, so this library should be directly bumped with the v2 update.

✔️ aiowaqi

-- Library: [aiowaqi](https://github.com/joostlek/python-waqi) Integrations using this library: [WAQI](https://github.com/home-assistant/core/tree/dev/homeassistant/components/waqi) Code owners: @joostlek - [x] Supports Pydantic v2 (PR: Initial commit) - [x] Is updated in HA (PR: #98000) Remarks:

❌ demetriek

-- Library: [demetriek](https://github.com/frenck/python-demetriek) Integrations using this library: [lametric](https://github.com/home-assistant/core/tree/dev/homeassistant/components/lametric) Code owners: @robbiet480 @frenck @bachya - [ ] Supports Pydantic v2 (PR: https://github.com/frenck/python-demetriek/pull/531) - [ ] Is updated in HA (PR: # ) Remarks:

✔️ elgato

-- Library: [elgato](https://github.com/frenck/python-elgato) Integrations using this library: [elgato](https://github.com/home-assistant/core/tree/dev/homeassistant/components/elgato) Code owners: @frenck - [x] Supports Pydantic v2 (PR: https://github.com/frenck/python-elgato/pull/833) - [x] Is updated in HA (PR: #102405 ) Remarks:

✔️ gcal-sync

-- Library: [gcal-sync](https://github.com/allenporter/gcal_sync) Integrations using this library: [Google Calendar](https://github.com/home-assistant/core/tree/dev/homeassistant/components/google) Code owners: @allenporter - [x] Supports Pydantic v2 (PR: https://github.com/allenporter/gcal_sync/pull/275) - [x] Is updated in HA (PR: #102010) Remarks: Supports shims. Needs a release so we can bump the dependency beforehand. Latest version also has ical 5.0.1, and that has pydantic v2 support since 5.0.0.

✔️ google-nest-sdm

-- Library: [google-nest-sdm](https://github.com/allenporter/python-google-nest-sdm) Integrations using this library: [Nest](https://github.com/home-assistant/core/tree/dev/homeassistant/components/nest) Code owners: @allenporter - [x] Supports Pydantic v2 (PR: https://github.com/allenporter/python-google-nest-sdm/pull/670) - [x] Is updated in HA (PR: #99175) Remarks:

✔️ ical

-- Library: [ical](https://github.com/allenporter/ical) Integrations using this library: [Local calendar](https://github.com/home-assistant/core/tree/dev/homeassistant/components/local_calendar) Code owners: @allenporter - [x] Supports Pydantic v2 (PR: https://github.com/allenporter/ical/pull/207) - [x] Is updated in HA (PR: #98998) Remarks:

❌ inflect

-- Library: [inflect](https://github.com/jaraco/inflect) Integrations using this library: [Abode](https://github.com/home-assistant/core/tree/dev/homeassistant/components/abode) Code owners: @shred86 - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks: This seems to be a deeper problem. I need to dive in to this to know the status.

❌ intellifire4py

-- Library: [intellifire4py](https://github.com/jeeftor/intellifire4py) Integrations using this library: [IntelliFire](https://github.com/home-assistant/core/tree/dev/homeassistant/components/intellifire) Code owners: @jeeftor - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

❌ lacrosse-view

-- Library: [lacrosse-view](https://github.com/IceBotYT/lacrosse_view) Integrations using this library: [LaCrosse View](https://github.com/home-assistant/core/tree/dev/homeassistant/components/lacrosse_view) Code owners: @IceBotYT - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

✔️ open-meteo

-- Library: [open-meteo](https://github.com/frenck/python-open-meteo) Integrations using this library: [Open Meteo](https://github.com/home-assistant/core/tree/dev/homeassistant/components/open_meteo) Code owners: @frenck - [x] Supports Pydantic v2 (PR: https://github.com/frenck/python-open-meteo/pull/353) - [x] Is updated in HA (PR: #103613) Remarks: To verify: There has been no code change to support it, but in the lock file is 2.0.3 so I am expecting this works. If this is as expected, it still needs a release and to be bumped.

❌ peco

-- Library: [peco](https://github.com/IceBotYT/peco-outage-api) Integrations using this library: [PECO Outage Counter](https://github.com/home-assistant/core/tree/dev/homeassistant/components/peco) Code owners: @IceBotYT - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks: There has been no explicit PR to fix support for v2. This might work but needs to be verified.

✔️ pvo

-- Library: [pvo](https://github.com/frenck/python-pvoutput) Integrations using this library: [PVOutput](https://github.com/home-assistant/core/tree/dev/homeassistant/components/pvoutput) Code owners: @frenck - [x] Supports Pydantic v2 (PR: https://github.com/frenck/python-pvoutput/pull/394) - [x] Is updated in HA (PR: #102398 ) Remarks:

❌ pyaussiebb

-- Library: [pyaussiebb](https://github.com/yaleman/pyaussiebb) Integrations using this library: [Aussie Broadband](https://github.com/home-assistant/core/tree/dev/homeassistant/components/aussie_broadband) Code owners: @nickw444 @Bre77 - [x] Supports Pydantic v2 ([commit](https://github.com/yaleman/pyaussiebb/commit/a0d240d75935955bd3af518d8e28fd5cac95722e)) - [ ] Is updated in HA (PR: #99077) Remarks: Library does not support v1 shims. Should be bumped with the v2 bump. cc @yaleman

✔️ pyrainbird

-- Library: [pyrainbird](https://github.com/allenporter/pyrainbird) Integrations using this library: [Rain Bird](https://github.com/home-assistant/core/tree/dev/homeassistant/components/rainbird) Code owners: @konikvranik @allenporter - [x] Supports Pydantic v2 (PR: https://github.com/allenporter/pyrainbird/pull/217) - [x] Is updated in HA (PR: #96610) Remarks: Still needs to update ical to =>5.0.0 as ical relies on pydantic as well.

❌ python-bsblan

-- Library: [python-bsblan](https://github.com/liudger/python-bsblan) Integrations using this library: [BSB-Lan](https://github.com/home-assistant/core/tree/dev/homeassistant/components/bsblan) Code owners: @liudger - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

✔️ python-kasa

-- Library: [python-kasa](https://github.com/python-kasa/python-kasa) Integrations using this library: [TP-Link Kasa Smart](https://github.com/home-assistant/core/tree/dev/homeassistant/components/tplink) Code owners: @rytilahti @thegardenmonkey - [x] Supports Pydantic v2 (PR: https://github.com/python-kasa/python-kasa/pull/504) - [x] Is updated in HA (PR: #103038) Remarks:

✔️ python-opensky

-- Library: [python-opensky](https://github.com/joostlek/python-opensky) Integrations using this library: [OpenSky Network](https://github.com/home-assistant/core/tree/dev/homeassistant/components/opensky) Code owners: @joostlek - [x] Supports Pydantic v2 (PR: https://github.com/joostlek/python-opensky/pull/85) - [x] Is updated in HA (PR: #97687) Remarks:

✔️ pytraccar

-- Library: [pytraccar](https://github.com/ludeeus/pytraccar) Integrations using this library: [Traccar](https://github.com/home-assistant/core/tree/dev/homeassistant/components/traccar) Code owners: @ludeeus - [ ] Supports Pydantic v2 (PR: ) - [x] Is updated in HA (PR: #103318) Remarks:

✔️ pyunifiprotect

-- Library: [pyunifiprotect](https://github.com/AngellusMortis/pyunifiprotect) Integrations using this library: [asd](https://github.com/home-assistant/core/tree/dev/homeassistant/components/unifiprotect) Code owners: @AngellusMortis @bdraco - [x] Supports Pydantic v2 (PR: https://github.com/AngellusMortis/pyunifiprotect/pull/297) - [x] Is updated in HA (PR: #96486) Remarks:

✔️ radios

-- Library: [radios](https://github.com/frenck/python-radios) Integrations using this library: [Radio Browser](https://github.com/home-assistant/core/tree/dev/homeassistant/components/radio_browser) Code owners: @frenck - [ ] Supports Pydantic v2 (PR: ) - [x] Is updated in HA (PR: #103614) Remarks:

❌ sfrbox-api

-- Library: [sfrbox-api](https://github.com/hacf-fr/sfrbox-api) Integrations using this library: [SFR Box](https://github.com/home-assistant/core/tree/dev/homeassistant/components/sfr_box) Code owners: @epenet - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

❌ systembridgeconnector

-- Library: [systembridgeconnector](https://github.com/timmo001/system-bridge) Integrations using this library: [System Bridge](https://github.com/home-assistant/core/tree/dev/homeassistant/components/system_bridge) Code owners: @timmo001 - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

✔️ tailscale

-- Library: [tailscale](https://github.com/frenck/python-tailscale) Integrations using this library: [asd](https://github.com/home-assistant/core/tree/dev/homeassistant/components/tailscale) Code owners: @frenck - [x] Supports Pydantic v2 (PR: https://github.com/frenck/python-tailscale/pull/370) - [ ] Is updated in HA (PR: # ) Remarks: To verify: There has been no code change to support it, but in the lock file is 2.0.3 so I am expecting this works. If this is as expected, it still needs a release and to be bumped.

✔️ vehicle

-- Library: [vehicle](https://github.com/frenck/python-vehicle) Integrations using this library: [RDW](https://github.com/home-assistant/core/tree/dev/homeassistant/components/rdw) Code owners: @frenck @joostlek - [x] Supports Pydantic v2 (PR: https://github.com/frenck/python-vehicle/pull/441) - [x] Is updated in HA (PR: #102379 ) Remarks:

✔️ withings-api

-- Library: [withings-api](https://github.com/vangorra/python_withings_api) Integrations using this library: [Withings](https://github.com/home-assistant/core/tree/dev/homeassistant/components/withings) Code owners: @vangorra - [ ] Supports Pydantic v2 (PR: ) - [x] Is updated in HA (PR: #101819) Remarks: Is now/will be replaced with a new lib without pydantic

❌ xbox-webapi

-- Library: [xbox-webapi](https://github.com/OpenXbox/xbox-webapi-python) Integrations using this library: [Xbox](https://github.com/home-assistant/core/tree/dev/homeassistant/components/xbox) Code owners: @hunterjm - [ ] Supports Pydantic v2 (PR: ) - [ ] Is updated in HA (PR: # ) Remarks:

✔️ yolink-api

-- Library: [yolink-api](https://github.com/YoSmart-Inc/yolink-api) Integrations using this library: [YoLink](https://github.com/home-assistant/core/tree/dev/homeassistant/components/yolink) Code owners: @matrixd2 - [x] Supports Pydantic v2 (PR: https://github.com/YoSmart-Inc/yolink-api/commit/836e61b8b755cd180291fbea7aef824fc12c2580) - [x] Is updated in HA (PR: #105124) Remarks:

✔️ youtubeaio

-- Library: [youtubeaio](https://github.com/joostlek/python-youtube) Integrations using this library: [YouTube](https://github.com/home-assistant/core/tree/dev/homeassistant/components/youtube) Code owners: @joostlek - [x] Supports Pydantic v2 (PR: ) - [x] Is updated in HA (PR: # ) Remarks:

✔️ zwave-js-server-python

-- Library: [zwave-js-server-python](https://github.com/home-assistant-libs/zwave-js-server-python) Integrations using this library: [Z-Wave](https://github.com/home-assistant/core/tree/dev/homeassistant/components/zwave_js) Code owners: @home-assistant/z-wave - [x] Supports Pydantic v2 (PR: https://github.com/home-assistant-libs/zwave-js-server-python/pull/703) - [x] Is updated in HA (PR: #94760) Remarks:

--

For the core members, feel free to update where needed. For others, let us know if something has updated in a comment and we will update it.

cc @cdce8p

frenck commented 10 months ago

As also reported on Discord. But replying here as well, as I was tagged. I have no intention of supporting v2 of Pydantic in my libraries.

Instead, I might be re-evaluating the use of Pydantic overall.

../Frenck

raman325 commented 10 months ago

@frenck if you do find a good alternative please do share it here. Others may want to choose the same option. I have v2 semi working but since all the internals have changed and are IMO more complicated I haven't been interested much in investing time into it

bachya commented 10 months ago

A good reminder: one can retain the V1 standards while upgrading to the V2 library. This is an excellent way to unstick this PR without committing yourself to a complete upgrade (assuming they don't get rid of this path at some point).

https://docs.pydantic.dev/latest/migration/#continue-using-pydantic-v1-features

frenck commented 10 months ago

As far as I am aware, it doesn't support everything backward. As said, I have no plans of supporting V2 at this point, and I am not willing to spend any second on it until I have explored other options.

../Frenck