mwhalen18 / sewage_DEP

A lightweight pipelining tool for data anlysis in R
https://mwhalen18.github.io/sewage/
Other
0 stars 0 forks source link

Add ability to pass data directly into pipeline #10

Open mwhalen18 opened 1 year ago

mwhalen18 commented 1 year ago

There should be a component that can pass arguments directly from one node to the next. This is an edge case, but sometimes you may want to pass a data object directly to a joiner (ie with no additional cleaning function in the middle. This might look something like

pipeline = Pipeline()
pipeline |>
     add_node(Chute(), name = "file1", input = file1) |>
     add_node(Chute(), name = "file2", input = file2) |>
     add_node(Joiner(method = "left_join"), name = "Joiner", input = c("file1", "file2")) 

Not sold on the name Chute() but this would make it easier to pass a data object directly into non-Node components (Splitters and Joiners)

mwhalen18 commented 1 year ago

In this edge case it would be possible to pass the data objects in the run command.

run(pipeline, file1 = df, file2 = df)

Until I see a real use case where this would be valid I am going to reject my own ideas here...