⬆️ Update `BaseSettings` import and `pydantic` dependency to v2.6

[GeigerJ2]

aiida-project currently requires pydantic~=1.10,>=1.10.8, which leads to dependency conflicts with the current, latest release of aiida-core (v2.5.1).

For instance, when one has a global aiida-core installation, e.g. via /usr/bin/python and tries to pip install aiida-project in editable mode to try out some local changes (naughty, I know), then the two packages aren't compatible due to the different pydantic requirements. I guess upgrading here would not be a bad idea anyway, as BaseSettings was moved to its own repository pydantic-settings. I'll see to resolve that soon.

[unkcpz]

Not sure related but when I use python3.9 I run into some typing errors rooted from pydantic:

  File "/home/jyu/Projects/WP-SSSP/aiida-project/aiida_project/commands/main.py", line 14, in <module>                                       
    from ..project import EngineType, load_project_class                                                                                                                                                                                                                                  
  File "/home/jyu/Projects/WP-SSSP/aiida-project/aiida_project/project/__init__.py", line 1, in <module>                                                                                                                                                                                  
    from .core import EngineType, ProjectDict, load_project_class                                                                            
  File "/home/jyu/Projects/WP-SSSP/aiida-project/aiida_project/project/core.py", line 8, in <module>                                         
    from .base import BaseProject                                                                                                            
  File "/home/jyu/Projects/WP-SSSP/aiida-project/aiida_project/project/base.py", line 22, in <module>                                        
    class BaseProject(BaseModel, ABC):                                                                                                       
  File "pydantic/main.py", line 178, in pydantic.main.ModelMetaclass.__new__                                                                 
  File "pydantic/typing.py", line 400, in pydantic.typing.resolve_annotations                                                                

  File "/home/jyu/micromamba/lib/python3.9/typing.py", line 292, in _eval_type                                                               
    return t._evaluate(globalns, localns, recursive_guard)                                                                                   
  File "/home/jyu/micromamba/lib/python3.9/typing.py", line 554, in _evaluate                                                                
    eval(self.__forward_code__, globalns, localns),                                                                                          
  File "<string>", line 1, in <module>                                                                                                                                                                                                                                                    
TypeError: unsupported operand type(s) for |: 'type' and 'type'  

So we either drop the support for py3.9 or pinning the pydantic version (if it is what cause the issue).

[GeigerJ2]

This seems like it should be fixed by adding: from __future__ import annotations as the | type annotation operator wasn't defined in Python 3.9. Had to add that to some of my code for aiida-core to make the Python 3,9 test suite run through. We can have a look to see if there are more issues like that. We should also write proper tests for the package, eventually :D

[mbercx]

I'm fine upgrading to pydantic v2.X, but dependency conflicts with aiida-core are not a good motivation. aiida-project is blessed in that it is an end-product that should be installed in its own separate environment (preferably with pipx) and never be a dependency of another package.

[GeigerJ2]

Fair enough, @mbercx. The issue arose from the fact that I'm not installing it via pipx, as the released version doesn't contain my beloved fish implementation. Instead, I cloned the repo and installed it locally via pip in my main OS Python. As I'm now using aiida-project as my one-stop shop manager for everything AiiDA-related, I keep aiida-core away from said main OS Python it. Though, it happened that I forgot to activate the dedicated venv and installed it there, thus leading to the conflict.

[mbercx]

Ah, but you should be able to just install a local directory via pipx as well. That said, then you are working globally with a development version, so that has risks for sure. ^^

[GeigerJ2]

Ah, good point, didn't know that as I never used pipx before. Well, it's not like we would ever break the package, e.g. by introducing circular imports or sthg :eyes: :sweat_smile:

[mbercx]

Re the typing issues, the from __future__ import annotations line is already in the base.py file:

https://github.com/aiidateam/aiida-project/blob/71dbc6b30a9ba94456dfa332d08efc491a15e6f7/aiida_project/project/base.py#L1

But I think the problem is that we also use such typing notation for the dir_structure field:

https://github.com/aiidateam/aiida-project/blob/71dbc6b30a9ba94456dfa332d08efc491a15e6f7/aiida_project/project/base.py#L26

And probably pydantic v1.X doesn't know how to deal with this. So that would be a good motivation to upgrade. 😁

[mbercx]

Seems upgrading also doesn't help immediately, but it does offer clear instructions:

TypeError: You have a type annotation 'dict[str, dict | list | Path] | list[Path] | Path' which makes use of newer typing features than are supported in your version of Python. To handle this error, you should either remove the use of new syntax or install the `eval_type_backport` package.