project-codeflare / codeflare

Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.
https://codeflare.dev
Apache License 2.0
218 stars 35 forks source link

Pipeline with a single dangling estimator node triggers an exception #27

Closed raghukiran1224 closed 3 years ago

raghukiran1224 commented 3 years ago

Describe the bug Possibly a corner case? ray-pipeline/codeflare/pipelines/Datamodel.py in get_pre_edges(self, node) 640 """ 641 pre_edges = [] --> 642 pre_nodes = self.__pre_graph__[node] 643 # Empty pre 644 if not pre_nodes:

KeyError: <codeflare.pipelines.Datamodel.EstimatorNode object at 0x7fa2d8920f10>

To Reproduce

## initialize codeflare pipeline by first creating the nodes
pipeline = dm.Pipeline()
node_a = dm.EstimatorNode('a', MinMaxScaler())
node_b = dm.EstimatorNode('b', StandardScaler())
node_c = dm.EstimatorNode('c', MaxAbsScaler())
node_d = dm.EstimatorNode('d', RobustScaler())

node_e = dm.AndNode('e', FeatureUnion())
node_f = dm.AndNode('f', FeatureUnion())
node_g = dm.AndNode('g', FeatureUnion())

## codeflare nodes are then connected by edges
pipeline.add_edge(node_a, node_e)
pipeline.add_edge(node_b, node_e)
pipeline.add_edge(node_c, node_f)
## node_d does not have a downstream node
# pipeline.add_edge(node_d, node_f)
pipeline.add_edge(node_e, node_g)
pipeline.add_edge(node_f, node_g)

pipeline_input = dm.PipelineInput()
xy = dm.Xy(X,y)
pipeline_input.add_xy_arg(node_a, xy)
pipeline_input.add_xy_arg(node_b, xy)
pipeline_input.add_xy_arg(node_c, xy)
pipeline_input.add_xy_arg(node_d, xy)

## execute the codeflare pipeline
pipeline_output = rt.execute_pipeline(pipeline, ExecutionType.FIT, pipeline_input)

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Smartphone (please complete the following information):

Additional context Add any other context about the problem here.

raghukiran1224 commented 3 years ago

closed per #31