search

grailbio / reflow

A language and runtime for distributed, incremental data processing in the cloud
Apache License 2.0
965 stars 52 forks source link

Compute a proper flow digest for switch continuations #119

Closed jcharum closed 4 years ago

jcharum commented 4 years ago

Compute a proper flow digest for switch expression evaluation continuations. The existing implementation was very incomplete and results in equivalent digests for non-equivalent expressions. The digest now includes the entire remainder of the switch expression to be evaluated.

Unit tests are added to verify.

This also contains a bit of refactoring to eliminate stutter ("evalSwitch.eval*").

jcharum commented 4 years ago

I did this in GitHub, because @mariusae reviewed the initial switch implementation and is still the world's leading expert on flow digests. However, @swami-m, I think we should also seek to be world's leading experts.

jcharum commented 4 years ago

@swami-m @prasadgopal: Could one of you have a look at this, as I think it would be good for someone at GRAIL to understand it? As it stands, switch caching is pretty broken.