Funsor computations rely on metadata stored as Funsor attributes (including .inputs, .output, .input_vars, .fresh, .bound) and rely on special behavior accessed via methods FunsorMeta.__call__() and Funsor.__call__(). One problem with relying on attributes and methods is that computations are then restricted to funsor-owned custom datatypes, namely subclasses of the base Funsor class. These restrictions make it cumbersome to perform funsor computations on non-Funsor immutable data such as tuples or frozensets of Funsors. Moreover these restrictions prevent interaction with 3rd-party libraries that define their own immutable datatypes.
Proposed solution
Consider moving all funsor metadata to a new metadata layer, say via a funsor.meta(-) function that looks up an immutable Metadata object with attributies .inputs, .output, .input_vars, .fresh, .bound etc. All Metadata instances are stored in a single global WeakKeyDictionary keyed on the immutable data which implements .__hash__().
Further consider supporting non-Funsor-types by replacing:
FunsorMeta.__call__() → reinterpret(-) or a new funsor.make(-), thereby supporting hash-consing and reinterpretation for non-Funsor-subclass types.
Funsor.__call__() → substitute(-, -) or Subs(-, -) or a new funsor.subs(arg, **kwargs), thereby supporting substitution for non-`Funsor.
This new type-agnostic interface for funsor data mechanics would be something like:
funsor.meta(-) to read metadata like .inputs, .output etc.
funsor.subs(-, **kwargs) to substitute for free variables in data
funsor.make(-) to create data in an interpretation-dependent way
Complications
str, tuple, and subclasses thereof do not support weak referencing: they cannot be used as keys in a WeakKeyDictionary and cannot be registered via weakref.finalize(-,-).
Tasks
[ ] Add a funsor.meta(-) function backed by a WeakKeyDictionary, and Metadata class (a namedtuple)
[ ] Move .inputs and .output into Metadata
[ ] Move .fresh and .bound into Metadata
[ ] Add a frozendict type
[ ] Compute metadata for other non-Funsor-subclass types: tuple, frozenset, frozendict
[ ] Simplify funsor.interpreter by using the new metadata
[ ] Add a funsor.subs(-, **kwargs) function (basically funsor.terms.substitute(-, -))
[ ] Refactor to use funsor.subs instead of Funsor.__call__()
[ ] Add a funsor.make(-) function
[ ] Refactor to use funsor.make(-) instead of FunsorMeta.__call__()
[ ] Support non-Funsor-subclass types in funsor.make(-)
[ ] Remove funsor.Terms.Tuple type since tuple now suffices
[ ] Implement a singledispatch function funsor.fingerprint(-) to extensibly hash non-hashable non-equality-comparable datatypes.
[ ] Consider renaming .inputs → .free_vars and .output → .domain or .dtype or similar
Alternative to #525
The problem
Funsor computations rely on metadata stored as
Funsorattributes (including.inputs,.output,.input_vars,.fresh,.bound) and rely on special behavior accessed via methodsFunsorMeta.__call__()andFunsor.__call__(). One problem with relying on attributes and methods is that computations are then restricted to funsor-owned custom datatypes, namely subclasses of the baseFunsorclass. These restrictions make it cumbersome to perform funsor computations on non-Funsorimmutable data such as tuples or frozensets ofFunsors. Moreover these restrictions prevent interaction with 3rd-party libraries that define their own immutable datatypes.Proposed solution
Consider moving all funsor metadata to a new metadata layer, say via a
funsor.meta(-)function that looks up an immutableMetadataobject with attributies.inputs,.output,.input_vars,.fresh,.boundetc. AllMetadatainstances are stored in a single globalWeakKeyDictionarykeyed on the immutable data which implements.__hash__().Further consider supporting non-
Funsor-types by replacing:FunsorMeta.__call__()→reinterpret(-)or a newfunsor.make(-), thereby supporting hash-consing and reinterpretation for non-Funsor-subclass types.Funsor.__call__()→substitute(-, -)orSubs(-, -)or a newfunsor.subs(arg, **kwargs), thereby supporting substitution for non-`Funsor.This new type-agnostic interface for funsor data mechanics would be something like:
funsor.meta(-)to read metadata like.inputs,.outputetc.funsor.subs(-, **kwargs)to substitute for free variables in datafunsor.make(-)to create data in an interpretation-dependent wayComplications
str,tuple, and subclasses thereof do not support weak referencing: they cannot be used as keys in aWeakKeyDictionaryand cannot be registered viaweakref.finalize(-,-).Tasks
funsor.meta(-)function backed by aWeakKeyDictionary, andMetadataclass (anamedtuple).inputsand.outputintoMetadata.freshand.boundintoMetadatafrozendicttypeFunsor-subclass types: tuple, frozenset, frozendictfunsor.interpreterby using the new metadatafunsor.subs(-, **kwargs)function (basicallyfunsor.terms.substitute(-, -))funsor.subsinstead ofFunsor.__call__()funsor.make(-)functionfunsor.make(-)instead ofFunsorMeta.__call__()Funsor-subclass types infunsor.make(-)funsor.Terms.Tupletype sincetuplenow sufficesfunsor.fingerprint(-)to extensibly hash non-hashable non-equality-comparable datatypes..inputs→.free_varsand.output→.domainor.dtypeor similar