[CONFIG] Resolve python requirements for python version and dependencies with server.

[frascuchon]

• Define python version 3.10?

-> >=3.9, <3.13

• Define core dependencies i.e. pydantic

Already defined in the SDK

• Be compatible with server

The server should be adapted to use pydantyc v2 which is not an evident task. Not sure if we can tackle this for v2.

[burtenshaw]

@jfcalvo Do you think you could tackle this one?

[jfcalvo]

@jfcalvo Do you think you could tackle this one?

I can take a look for sure.

[jfcalvo]

As @frascuchon commented we can change Python versions but we will still have problems with pydantic version on argilla-server.

I have used pip check to show the current problems we have trying to install argilla-sdk and argilla-server in the same Python environment.

From argilla-sdk:

argilla-sdk$ pip check
argilla-server 2.0.0.dev0 requires pydantic<2.0,>=1.10.7, but you have pydantic 2.7.3 which is incompatible.

From argilla-server:

argilla-server$ pip check
argilla-sdk 2.0.0a0.dev0 has requirement pydantic<3.0.0,>=2.6.0, but you have pydantic 1.10.15.

[burtenshaw]

Thanks @jfcalvo. How feasible would it be to upgrade the server to pydantic v2?

Pydantic v1 features are exposed via the v2 api: https://docs.pydantic.dev/latest/migration/#continue-using-pydantic-v1-features

@frascuchon explored this before we deprecated the argilla api v1 endpoints. Maybe it's easier now...

[jfcalvo]

@burtenshaw It's something we can explore but not sure if it will be possible inside the schedule with have for releasing v2.0.

[frascuchon]

Thanks @jfcalvo. Using pydantic v1 from v2 wouldn't be a problem. The problem is using those with FastAPI which expects v2 models.

We need to migrate all the schemas at once. (it would be nice if we could migrate endpoints iteratively, but it's not possible). Several changes imply a deep endpoint revision:

• The pattern regex is not supported anymore. We probably need to have custom validators for regex
• The default values None for Optionals are not supported anymore. We need to explicitly set =None for those models where applies.

These changes and probably some more would be a potential nest of bugs if we don't take the time to validate everything.

[burtenshaw]

Thanks @frascuchon. It's useful to have that transition written out.

What would you suggest for this issue then?

[burtenshaw]

@jfcalvo @frascuchon As discussed, it's not feasible to solve this within the current time frame, and there's minimal value to the end user because they can run server and client in docker of spaces. Therefore, we should solve this outside of implementation by adding detailed docs with warnings that make it clear that the server and sdk cannot be run in the same python env due to a pydantic conflict.

We could support this further by checking the metrics on python server users, remove server dependence in argilla-sdk.

@nataliaElv We can focus on this v2.2.