Pydantic Validation error on import of coiled.prefect Credentials

[elementace]

I;m trying to follow the basic setup of coiled and prefect authentication as detailed here: https://docs.coiled.io/user_guide/labs/prefect.html

However I keep getting pydantic validation errors when I try to import Credentials.

To recreate the issue, in MacOS 14.3.1 shell I:

conda create -n coiledtest
conda activate coiledtest
python -V  - > 3.12.3
conda install pip
pip install coiled prefect
python -c "from coiled.prefect import Credentials" 

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/coiled/prefect.py", line 13, in <module>
    class Credentials(Block):
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/main.py", line 197, in __new__
    fields[ann_name] = ModelField.infer(
                       ^^^^^^^^^^^^^^^^^
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/fields.py", line 504, in infer
    return cls(
           ^^^^
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/fields.py", line 434, in __init__
    self.prepare()
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/fields.py", line 555, in prepare
    self.populate_validators()
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/fields.py", line 829, in populate_validators
    *(get_validators() if get_validators else list(find_validators(self.type_, self.model_config))),
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/validators.py", line 765, in find_validators
    raise RuntimeError(f'no validator found for {type_}, see `arbitrary_types_allowed` in Config')
RuntimeError: no validator found for <class 'pydantic.types.SecretStr'>, see `arbitrary_types_allowed` in Config

coiled == 1.19.0 prefect == 2.18.0 pydantic == 2.7.1 pydantic_core == 2.18.2

I've tried rolling back to various permutations of minor and major versions for these libraries but haven't been able to successfully import yet.

[ntabris]

@elementace thanks for raising this issue! I've noticed the prefect docs say we need to use the pydantic v1 version of SecretString. We'll get a fix out in the next few days.

[elementace]

Amazing, thanks for fixing this up so quick team!

[elementace]

@ntabris @dan-blanchard Just tried

python -c "from coiled.prefect import Credentials"

with

coiled-1.19.2.dev5 
prefect-2.18.1 
pydantic-2.7.1 
pydantic-core-2.18

and got a slightly different error. Do you see this too?

TypeError: Object of type 'FieldInfo' is not JSON serializable

Full traceback:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/coiled/prefect.py", line 20, in <module>
    class Credentials(Block):
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/main.py", line 282, in __new__
    cls = super().__new__(mcs, name, bases, new_namespace, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen abc>", line 106, in __new__
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/prefect/utilities/dispatch.py", line 100, in _register_subclass_of_base_type
    register_type(cls)
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/prefect/utilities/dispatch.py", line 154, in register_type
    key = get_dispatch_key(cls)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/prefect/utilities/dispatch.py", line 77, in get_dispatch_key
    dispatch_key = dispatch_key()
                   ^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/prefect/blocks/core.py", line 315, in __dispatch_key__
    return block_schema_to_key(cls._to_block_schema())
                               ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/prefect/blocks/core.py", line 487, in _to_block_schema
    fields = cls.schema()
             ^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/main.py", line 664, in schema
    s = model_schema(cls, by_alias=by_alias, ref_template=ref_template)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/schema.py", line 188, in model_schema
    m_schema, m_definitions, nested_models = model_process_schema(
                                             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/schema.py", line 581, in model_process_schema
    m_schema, m_definitions, nested_models = model_type_schema(
                                             ^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/schema.py", line 622, in model_type_schema
    f_schema, f_definitions, f_nested_models = field_schema(
                                               ^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/schema.py", line 248, in field_schema
    s, schema_overrides = get_field_info_schema(field)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/schema.py", line 216, in get_field_info_schema
    schema_['default'] = encode_default(field.default)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/schema.py", line 995, in encode_default
    return pydantic_encoder(dft)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elementace/opt/miniconda3/envs/coiledtest/lib/python3.12/site-packages/pydantic/v1/json.py", line 90, in pydantic_encoder
    raise TypeError(f"Object of type '{obj.__class__.__name__}' is not JSON serializable")
TypeError: Object of type 'FieldInfo' is not JSON serializable

Rolling back to this (pydantic 1) has fixed the errors for me: coiled 1.17 prefect 2.16 pydantic 1.10.15