microsoft / knossos-ksc

Compiler with automatic differentiation
Other
45 stars 10 forks source link

Refactor how rewrite.py passes delayed match application (for typing & clarity) #921

Closed cgravill closed 3 years ago

cgravill commented 3 years ago

Follow on from #920

Instead of creating a Match with a variable number of custom parameters then dispatched by an ABC, then filling in kwargs. Can we instead just create a callable and let it be run on demand?

The benefit is that it keeps the parameter handling local, so could do away with lots of the comments. It's going to type check, which the "specialisation" of kwargs isn't accepted by mypy, even if we add kwargs to the overriding functions.

cgravill commented 3 years ago

In short, yes :). I quite like the lambda! (And it certainly beats the double class hierarchy in RLO).

Super, I'll keep going to make this possibly correct!

I think it's nice for the Match to contain the actual Rule(Matcher) object as a field, just for reference. But we can put the callable alongside that. Also, I think the "rule_specific_data" in the Match is no longer needed - it is embedded within the Callable ?

Yes, it's embedded. Plan would be to do away with that entirely if this works.

The original design gave each RuleMatcher subclass an "apply_at" method partly because it seemed like a nice division, too - between (a) finding matches and (b) doing the rewrites that were found - but removing the abstractmethod in the superclass might be enough to make mypy happy, with each subclass:

  • declaring it's own differently-typed apply_at
  • creating Callable's that call only that class's apply_at, with appropriate arguments

and that might constitute a smaller change, that still captures the essential bit with the Callable?

Also probably good to separate out from the other typechecking changes to follow.

Agreed, I hope to merge #920 to master soon, then to here. I could have done this change separately which might have been simpler to review.

cgravill commented 3 years ago

OK, tests pass and it's tidied up. More of it type checks but still further changes, I agree with the suggestion to get this resolved before more of those.

cgravill commented 3 years ago

Very nice. One generic style suggestion.

Thanks @awf I think this is now ready, I'll merge it later today if there's no further refinement

cgravill commented 3 years ago

For reference, for mypy with rewrites.py we now down to:

src\python\ksc\rewrites.py:219: error: Need type annotation for "possible_filter_terms"
src\python\ksc\rewrites.py:225: error: "Expr" has no attribute "name"
src\python\ksc\rewrites.py:228: error: "Expr" has no attribute "name"
src\python\ksc\rewrites.py:274: error: Item "List[Var]" of "Union[Var, List[Var]]" has no attribute "name"
src\python\ksc\rewrites.py:339: error: Missing positional argument "var_names_to_exprs" in call to "visit" of "SubstPattern"
src\python\ksc\rewrites.py:339: error: Argument 1 to "visit" of "SubstPattern" has incompatible type "Expr"; expected "SubstPattern"
src\python\ksc\rewrites.py:339: error: Argument 2 to "visit" of "SubstPattern" has incompatible type "Optional[Mapping[str, Expr]]"; expected "Expr"
src\python\ksc\rewrites.py:341: error: Missing positional argument "var_names_to_exprs" in call to "visit" of "SubstPattern"
src\python\ksc\rewrites.py:341: error: Argument 1 to "visit" of "SubstPattern" has incompatible type "Expr"; expected "SubstPattern"
src\python\ksc\rewrites.py:341: error: Argument 2 to "visit" of "SubstPattern" has incompatible type "Optional[Mapping[str, Expr]]"; expected "Expr"
src\python\ksc\rewrites.py:366: error: "Mapping[str, Expr]" has no attribute "update"
src\python\ksc\rewrites.py:388: error: Incompatible types in assignment (expression has type "Optional[Mapping[str, Expr]]", variable has type "Dict[Any, Any]")
src\python\ksc\rewrites.py:466: error: Incompatible return value type (got "Tuple[Union[Expr, Any], Mapping[str, Expr]]", expected "Tuple[Var, Mapping[str, Expr]]")
src\python\ksc\rewrites.py:481: error: Signature of "visit" incompatible with supertype "ExprVisitor"
src\python\ksc\rewrites.py:485: error: "None" has no attribute "type_"
src\python\ksc\rewrites.py:487: error: "None" has no attribute "type_"
src\python\ksc\rewrites.py:491: error: Incompatible return value type (got "None", expected "Expr")
src\python\ksc\rewrites.py:493: error: Signature of "visit_var" incompatible with supertype "ExprTransformer"
src\python\ksc\rewrites.py:493: error: Signature of "visit_var" incompatible with supertype "ExprVisitor"
src\python\ksc\rewrites.py:496: error: Signature of "visit_let" incompatible with supertype "ExprTransformer"
src\python\ksc\rewrites.py:496: error: Signature of "visit_let" incompatible with supertype "ExprVisitor"
src\python\ksc\rewrites.py:510: error: Signature of "visit_lam" incompatible with supertype "ExprTransformer"
src\python\ksc\rewrites.py:510: error: Signature of "visit_lam" incompatible with supertype "ExprVisitor"

which will continue in future prs.