For uniformity of input output to pipeline stages, we need to have a list of Future references to Xy objects. Since Ray remote calls always return a future reference and we return a holder class with reference to X and y, which are compute tasks that can be in flight. A list of references allows for the parallelization of operations.
Uniformity of data exchange is enabled by choosing a list of future references as the way to exchange data.
Acceptance Criteria
[x] Move away from current implementation to a future references implementation
Overview
For uniformity of input output to pipeline stages, we need to have a list of
Future
references toXy
objects. Since Ray remote calls always return a future reference and we return a holder class with reference to X and y, which are compute tasks that can be in flight. A list of references allows for the parallelization of operations.Uniformity of data exchange is enabled by choosing a list of future references as the way to exchange data.
Acceptance Criteria
Questions
Assumptions
Reference