mwhalen18 / sewage_DEP

A lightweight pipelining tool for data anlysis in R
https://mwhalen18.github.io/sewage/
Other
0 stars 0 forks source link

`run` should pass ellipses to first node #5

Closed mwhalen18 closed 1 year ago

mwhalen18 commented 1 year ago

When running a pipeline, the ellipses should be passed to the first node in the pipeline as additional arguments.

Currently, to do this you would need to specify

pipeline = Pipeline()
pipeline = pipeline |>
     add_node(component = DBI::db_get_query, name = "Query", input = "connection", query = query)
run(pipeline, connection = con)

An optimal solution would look like

pipeline = Pipeline()
pipeline = pipeline |>
     add_node(component = DBI::db_get_query, name = "Query", input = "connection")
run(pipeline, connection = con, query = query)

This would require run to identify the first node in the pipeline and pass the ... when running execute

mwhalen18 commented 1 year ago

Closing this. Instead, the ellipses in run will be reserved for arguments to pass as inputs to all entrypoints in the pipeline. This was a fun thought though, but practically it makes sense to keep the API for the nodes the same regardless of where they are in the pipeline.