open_esm_datastore root_validator error

[andreall]

Hi,

I am trying to load CMIP6 data on a newly pangeo environment on a linux machine.

I tried:

url="https://storage.googleapis.com/cmip6/pangeo-cmip6.json"
col=intake.open_esm_datastore(url)

I got:

---------------------------------------------------------------------------
PydanticUserError                         Traceback (most recent call last)
Cell In[13], line 2
      1 url="https://storage.googleapis.com/cmip6/pangeo-cmip6.json"
----> 2 col=intake.open_esm_datastore(url)

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/intake/__init__.py:66, in __getattr__(attr)
     64 if attr[:5] == "open_":
     65     if attr[5:] in registry.drivers.enabled_plugins():
---> 66         driver = registry[attr[5:]]  # "open_..."
     67         return driver
     68     else:

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/intake/source/__init__.py:32, in DriverRegistry.__getitem__(self, item)
     30 it = self.drivers.enabled_plugins()[item]
     31 if isinstance(it, entrypoints.EntryPoint):
---> 32     return it.load()
     33 elif isinstance(it, str):
     34     return import_name(it)

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/entrypoints.py:79, in EntryPoint.load(self)
     76 def load(self):
     77     """Load the object to which this entry point refers.
     78     """
---> 79     mod = import_module(self.module_name)
     80     obj = mod
     81     if self.object_name:

File ~/micromamba/envs/pangeo/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    124             break
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:992, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/intake_esm/__init__.py:9
      6 from pkg_resources import DistributionNotFound, get_distribution
      8 from . import tutorial
----> 9 from .core import esm_datastore
     10 from .derived import DerivedVariableRegistry, default_registry
     11 from .utils import set_options, show_versions

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/intake_esm/core.py:21
     18 from fastprogress.fastprogress import progress_bar
     19 from intake.catalog import Catalog
---> 21 from .cat import ESMCatalogModel
     22 from .derived import DerivedVariableRegistry, default_registry
     23 from .source import ESMDataSource

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/intake_esm/cat.py:68
     64         validate_all = True
     65         validate_assignment = True
---> 68 class Assets(pydantic.BaseModel):
     69     column_name: pydantic.StrictStr
     70     format: typing.Optional[DataFormat]

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/intake_esm/cat.py:78, in Assets()
     74     validate_all = True
     75     validate_assignment = True
     77 @pydantic.root_validator
---> 78 def _validate_data_format(cls, values):
     79     data_format, format_column_name = values.get('format'), values.get('format_column_name')
     80     if data_format is not None and format_column_name is not None:

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/pydantic/deprecated/class_validators.py:222, in root_validator(pre, skip_on_failure, allow_reuse, *__args)
    212 warn(
    213     'Pydantic V1 style `@root_validator` validators are deprecated.'
    214     ' You should migrate to Pydantic V2 style `@model_validator` validators,'
   (...)
    217     stacklevel=2,
    218 )
    220 if __args:
    221     # Ensure a nice error is raised if someone attempts to use the bare decorator
--> 222     return root_validator()(*__args)  # type: ignore
    224 if allow_reuse is True:  # pragma: no cover
    225     warn(_ALLOW_REUSE_WARNING_MESSAGE, DeprecationWarning)

File ~/micromamba/envs/pangeo/lib/python3.10/site-packages/pydantic/deprecated/class_validators.py:228, in root_validator(pre, skip_on_failure, allow_reuse, *__args)
    226 mode: Literal['before', 'after'] = 'before' if pre is True else 'after'
    227 if pre is False and skip_on_failure is not True:
--> 228     raise PydanticUserError(
    229         'If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`.'
    230         ' Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.',
    231         code='root-validator-pre-skip',
    232     )
    234 wrap = partial(_decorators_v1.make_v1_generic_root_validator, pre=pre)
    236 def dec(f: Callable[..., Any] | classmethod[Any, Any, Any] | staticmethod[Any, Any]) -> Any:

PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.

For further information visit https://errors.pydantic.dev/2.0.2/u/root-validator-pre-skip

I have installed with micromamba and have:

• Python 3.10.12
• intake 0.7.0
• intake-esm 2023.6.14

I have trid and it works without any issue on another machine (MacOS) where I work with anaconda:

• python 3.8.13
• intake 0.6.7
• intake-esm 2022.9.8

Any thoughts? thanks!

[aulemahal]

This is probably due to an incompatibility in intake-esm with the latest pydantic. ~Can you try downgrading pydantic < 2 ?~

EDIT : The issue has already been solved in intake-esm v2023.07.07, try to install this newer version if possible!

[andreall]

it worked with the intake-esm update! thanks a lot!