jcrist / msgspec

A fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML
https://jcristharif.com/msgspec/
BSD 3-Clause "New" or "Revised" License
2.3k stars 67 forks source link

Allow use of `object.__setattr__` to modify `frozen` instances of `Struct` #588

Closed wikiped closed 10 months ago

wikiped commented 10 months ago

Description

Docs for built-in dataclass mention using object.__setattr__ to modify frozen instances.

When the same approach is used to modify frozen Struct (for example in __post_init__) this leads to error:

TypeError: can't apply this __setattr__ to <StructName> object

At the same time Struct api docs say:

frozen (bool, default False) – Whether instances of this type are pseudo-immutable. If true, attribute assignment is disabled ...

So, pseudo-immutable in the docs 'hints' that there is a way to modify the struct, but it is unclear how exactly this could be done, since Struct.__setattr__ is hardcoded to unconditionally throw error unlike builtin dataclass.

jcrist commented 10 months ago

Here "pseudo-immutable" was only intended to indicate that struct attributes couldn't be altered after creation, but if the attribute values themselves weren't immutable they could still be mutated. For example:

class Demo(Struct, frozen=True):
    x: Any

obj = Demo([])

# Can't change x; this will error
obj.x = 2

# Can mutate the value of x if x itself is mutable; this won't error
obj.x.append(2)

I agree that the docs here aren't being specific in what we mean by that, I'd be happy to update them (or accept a PR to update them) to be more clear.

When the same approach is used to modify frozen Struct (for example in __post_init__) this leads to error:

The error you're seeing is raised by the cpython VM since due to layout differences object.__setattr__ can't apply to struct instances. Like anything in cpython I'm sure there's some way to hack around the immutability of a struct (with ctypes anything is possible), but we don't provide or document a specific method to do so. Messing with immutability guarantees can lead to weird unexpected behavior, i'd rather not enable it unless necessary. Can you explain your use case here?

wikiped commented 10 months ago

The error you're seeing is raised by the cpython VM since due to layout differences object.setattr can't apply to struct instances.

I understand the origin of the error, but was hoping for a "way-around" this "limitation".

Can you explain your use case here?

class A(msgspec.Struct, frozen=False):
    name: str
    alias: str | None = None

    def __post_init__(self):
        if not self.alias:
            self.alias = self.name

What it basically comes down to - is setting some value for a field, depending on the value of the other field of the struct.

Of course, this logic could be applied at a consumer end. But the same could, probably, be said about everything else linked to validation.

This isn't a 'deal breaker', but a nice-to-have feature for 'conscenting adults'.

jcrist commented 10 months ago

I've added msgspec.structs.force_setattr to handle this task. It works the same as setattr, but will still work on a frozen struct. This does violate the frozen guarantees, so should only be used when you know what you're doing. Fixed in #600.

wikiped commented 10 months ago

Thank you very much for adding this feature!