Implement general (and generalized) assignment using reified LValues

[EvanKirshenbaum]

(Splitting off #99)

This started by wanting to add modifiers (+=, *=, etc.) to the macro language. It was straightforward to modify the assignment rules to replace the ASSIGN token with an assign_op rule which had an associated function name (e.g., ADD for +=.

Rather than modify the two current assignment rules (for names (variables and special variables) and attributes), my plan it to have a single assignment rule of the form

   <assoc=right> lhs=expr op=assign_op rhs=expr

To support this, name and attribute expressions will return lval types (unless they are non-settable), and the actual values returned will be subclasses of LValue, which is currently defined as

class LValue(ABC):
    val_type: Final[Type]

    def __init__(self, val_type: Type):
        self.val_type = val_type.lval
    @abstractmethod
    def get_value(self) -> Any:
        ...
    @abstractmethod
    def _set(self, val: Any) -> Any: # @UnusedVariable
        ...
    def set_value(self, val: Any) -> Any:
        if self.is_short_circuit_val(val):
            return val
        return self._set(val)
    def is_short_circuit_val(self, val: Any) -> bool: # @UnusedVariable
        return False
    def modify(self, *, delta: Callable[[], Delayed[Any]],
               modifier: Callable[[Any, Any], Delayed[Any]]) -> Delayed[Any]:

        old = self.get_value()
        if self.is_short_circuit_val(old):
            return Delayed.complete(old)

        def with_delta(d: Any) -> Delayed[Any]:
            if self.is_short_circuit_val(d):
                return Delayed.complete(d)

            return (modifier(old, d)
                    .transformed(lambda new: self.set_value(new)))
        return (delta()
                .chain(with_delta))

where modify's delta computes the rhs of the modifying assignment and modifier computes the modification from the old and new values. LVAL types are considered to be subtypes of their RVAL equivalents and convert by calling get_value().

Having assignment as a single rule that expects an lval type as its lhs means that it will be straightforward for it to work when we add things like lists and records.

Variables can be defined as

class DMFLvalue(LValue):
    def is_short_circuit_val(self, val:Any)->bool:
        return isinstance(val, EvaluationError)

class Variable(DMFLvalue):
    name: Final[str]
    env: Final[Environment]

    def __init__(self, name: str, var_type: Type, env: Environment) -> None:
        super().__init__(var_type)
        self.name = name
        self.env = env
    def get_value(self) -> Any:
        name = self.name
        val = self.env.lookup(name)
        if val is None:
            return UninitializedVariableError(f"Uninitialized variable: {name}")
        return val
    def _set(self, val:Any) -> Any:
        self.env.define(self.name, val)
        return val

and special variable lvals would similarly bind the environment. In attribute expressions, the lval would bind the object whose attribute was being taken.

The above definition works fine for variables, which only need to look in or modify the bound environment, but it's possible that for some attributes (which bind the object) and special variables (which bind the environment), they may not be able to return immediately (#129), so it would probably be best if get_value() and set_value() actually return Delayed objects. (Which was actually my first cut implementation, but I talked myself out of it.)

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jun 04, 2022 at 10:07 AM PDT.

[EvanKirshenbaum]

One problem with this is that for special variables and attributes, it may be desirable for them to be set to values other than their value type. For example

clock.speed = 100ms;
clock.speed = 10Hz;
clock.speed = 10 ticks/second;

The problem is that the assignment visitor needs to be able to tell from the type (alone) of the LHS what the allowed types of the RHS are (and what conversions might be necessary), and the LValue.set_value() method needs to be told what the type of its argument is so that it can correctly interpret its value argument and pick the right action.

As currently implemented,

class Type:
    ...
    _lval: Optional[LvalType] = None
    @property
    def lval(self) -> LvalType:
        lt = self._lval
        if lt is None:
            lt = self._lval = LvalType(self)
        return lt
    ...

class LvalType(Type):
    ....
    @property
    def lval(self) -> LvalType:
        return self
    ...

This will probably have to be changed to having an lval_with_setters_for(types...) method that caches values in a local dict to preserve identity, with LvalType being able to tell the visitor the types it can accept. (By default, this will just be the rval type.)

Note that these types may be hierarchical (e.g., an attribute that does different things when set to a FLOAT and an INT or, more likely, a STRING and an ANY), so the visitor will have to find the narrowest type that works.

To support all of this, attributes (not bound attribute values) will want to provide caching lookup methods that give the lval types that apply for specific object types. And the lval types themselves will want to provide (and cache) the value type to use for various rhs expression types in assignments. Alternatively, the assignment provides its value (after confirming with the lval type that it's acceptable) along with its type, and the lvalue finds the correct setter, caching in the attribute or special form which one is to be used for each provided type, along with the required conversion.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jun 04, 2022 at 10:44 AM PDT.

[EvanKirshenbaum]

As noted in https://github.com/HPInc/HP-Digital-Microfluidics/issues/151 [comment by @EvanKirshenbaum on Jun 05, 2022 at 4:34 PM PDT], if we have maybe types, the getter will want to take the desired type so that it can know to generate a MaybeNotSatisfied error if it's a missing value and the desired type is an rval. We can probably do this up in LValue itself, with a call-down method to get the actual error value.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jun 05, 2022 at 4:36 PM PDT.