taoensso / nippy

The fastest serialization library for Clojure
https://www.taoensso.com/nippy
Eclipse Public License 1.0
1.04k stars 60 forks source link

Thawing var leads to error #143

Closed aneilbaboo closed 11 months ago

aneilbaboo commented 2 years ago

When freezing and thawing a nested structure, is there a way to turnoff metadata for one particular type?

I have a tree structure which includes clojure.lang.Var. As @atdixon pointed out here, thawing a Var results in an error when nippy attempts to apply metadata to it.

I know about *incl-metadata* and the option to freeze, but that forces me to turn of metadata persistence for all objects in a tree. Given that some classes like Var don't play well with metadata, it would be nice to have a way to provide a metadata exclusion list.

ptaoussanis commented 2 years ago

@aneilbaboo Hi Aneil, thanks for the clear+motivated question 👍

A couple quick thoughts:

(extend-protocol taoensso.nippy/IFreezable2
    clojure.lang.Var
    (taoensso.nippy/-freeze-with-meta! [x data-output]
      (taoensso.nippy/-freeze-without-meta! x data-output)))

This basically instructs Nippy to handle clojure.lang.Var's -freeze-with-meta! with -freeze-without-meta!.

This would only be a temporary workaround since it's making use of an implementation detail (i.e. would be subject to breakage) - but the data produced by such freezing would be forward compatible.

I.e. the mechanism to achieve the goal is subject to breakage, but the resulting data would be safe (not subject to breakage).

As for providing a general option for turning off metadata persistence for specific types: am open to the idea in principle, but would want to first understand a little better what the common use cases would be. There's several different ways that this could be implemented - and there'd be differences in convenience, flexibility, and performance.

One obvious (and high-performance) way would be to just add a public fn to do something like the workaround above for a given class: i.e. just register an IFreezable2 implementation for the given class that just skips metadata.

An alternative that's more flexible but slower would be for *incl-metadata?* to support a predicate value, etc.

What's your specific use case for de/serializing Vars?

aneilbaboo commented 2 years ago

I like your suggestion of testing whether an object supports IObj before calling with-meta on it. Although there is no guarantee in the docs that IObj is necessary for successfully calling with-meta, the code shows with-meta is just a wrapper around it.

I'm saving dynamic variable bindings to regenerate later. I worked around this by saving symbol names instead. Thanks for the extend-protocol suggestion.

ptaoussanis commented 11 months ago

Closing, this will be addressed in next Nippy release.