field "model_flags" conflicts with Pydantic 2 protected namespaces

dnewood commented 9 months ago

Environment

DiffSync version: 2.0.0
Python version Python 3.10.12

Observed Behavior

Getting warning logs from Pydantic about protected namespace violations within the DiffSync library

Expected Behavior

No logging output at the warning level

Steps to Reproduce

python3 -i
from diffsync import DiffSyncModel

Extra Information

Pydantic introduces protected namespaces in their new documentation that places warning logs for things like model_ fields documentation here

potential workaround is to use the ConfigDict to set protected_namespaces to an empty value.

class Device(DiffSyncModel):
    """common model used for Diffsync"""

    # Disables new Pydantic v2 protections since diffsync uses model_ fields
    model_config = ConfigDict(
        protected_namespaces=()
    )

However, this does not prevent the log entries from the diffsync BaseModel here: https://github.com/networktocode/diffsync/blob/13f5150d66ec76b637f56c908d1ab300bf63661d/diffsync/__init__.py#L110

nrnvgh commented 3 months ago

I just tripped over this as well. FWIW, I was able to suppress the warnings by placing this above the Adapter import:

#
# Suppress pydantic warnings
import warnings
warnings.filterwarnings(    # pylint: disable=wrong-import-position
    action="ignore",
    category=UserWarning,
    module="pydantic",
)

#
# Normal code resumes
from diffsync import Adapter

Obvs this should be fixed Correctly™ in diffsync, but at least this silences things.

dylanbob commented 2 months ago

Hi. With v3.0.0, nautobot-app-ssot now moves to diffsync 2.0, making this warning appear and slowing down the process A LOT in nautobot. Any fix in perspective?

Kircheneer commented 2 months ago

slowing down the process A LOT in nautobot.

Can you elaborate what you mean with this? The warning itself shouldn't slow down anything. Can you provide an example with timings?

dylanbob commented 2 months ago

This might be an illusion, but I felt like the very verbose output in the debug console slowed down the process. I do not have any data to back this up, sorry.

jdrew82 commented 2 months ago

@Kircheneer this is related to this error message:

/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_flags" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

jamesharr commented 2 months ago

I've tripped over this as well.

IMO:

The best thing to do here is to respect the Pydantic protected namespaces and choose another field name for model_flags. Maybe instance_flags though I'm very open to alternatives -- that's just the first thing that came to mind.
A patch should be backwards compatible with model_flags, but provide a deprecation warning
I personally would let SemVer guarantees slide a bit since the 2.0 release is so fresh, especially if there's a deprecation warning somewhere.

jamesharr commented 3 weeks ago

I did a quick POC of how DiffSync could rename model_flags, keep backwards compatibility, but also generate a useful deprecation warning for DiffSync users.

import warnings
from pydantic import BaseModel, model_validator

class DiffSyncModel(BaseModel):
    instance_flags: str  # NB: use real-type, this is just a place-holder for the proof of concept

    @model_validator(mode="before")
    @classmethod
    def _translate_model_flags(cls, data:any) -> any:
        if isinstance(data, dict) and "model_flags" in data:
            data["instance_flags"] = data["model_flags"]
            del data["model_flags"]
            warnings.warn("model_flags is deprecated, use instance_flags instead", DeprecationWarning,)
        return data

    @property
    def model_flags(self):
        warnings.warn("model_flags is deprecated, use instance_flags instead", DeprecationWarning,)
        return self.instance_flags

    @model_flags.setter
    def model_flags(self, value):
        warnings.warn("model_flags is deprecated, use instance_flags instead", DeprecationWarning,)
        self.instance_flags = value

print("=== No warnings ===")
obj1 = DiffSyncModel(instance_flags="foo")
print(obj1.instance_flags)
obj1.instance_flags = "bar"

print("=== Generates warning at instantiation time ===")
obj2 = DiffSyncModel(model_flags="asdf")

print("=== Generates warning on read ===")
obj3 = DiffSyncModel(instance_flags="foo")
print(obj3.model_flags)

print("=== Generates warning on write ===")
obj4 = DiffSyncModel(instance_flags="foo")
obj1.model_flags = "bar"

This is what's generated when I run the example code:

% python3 dep-warn.py
=== No warnings ===
foo
=== Generates warning at instantiation time ===
/Users/jharr/Downloads/dep-warn.py:13: DeprecationWarning: model_flags is deprecated, use instance_flags instead
  warnings.warn("model_flags is deprecated, use instance_flags instead", DeprecationWarning,)
=== Generates warning on read ===
/Users/jharr/Downloads/dep-warn.py:18: DeprecationWarning: model_flags is deprecated, use instance_flags instead
  warnings.warn("model_flags is deprecated, use instance_flags instead", DeprecationWarning,)
foo
=== Generates warning on write ===
/Users/jharr/Downloads/dep-warn.py:23: DeprecationWarning: model_flags is deprecated, use instance_flags instead
  warnings.warn("model_flags is deprecated, use instance_flags instead", DeprecationWarning,)

The only thing I'm not wild about is using a model_validator in a library like this.

networktocode / diffsync