ropensci / drake

An R-focused pipeline toolkit for reproducibility and high-performance computing
https://docs.ropensci.org/drake
GNU General Public License v3.0
1.34k stars 129 forks source link

How to mix cross() and map()? #1351

Closed potash closed 3 years ago

potash commented 3 years ago

Prework

Question

How to branch in a way that combines cross() and map()? I have three arguments but two of them are paired. For example arg1 can be "a" or "b", arg2 can be 1 or 2 and arg3 can be "red" or "blue" and I want to branch the following combinations:

a, 1, red a, 2, blue b, 1, red b, 2, blue

Reproducible example

test_plan = drake_plan(
  test = target(
    print(c(arg1, arg2, arg3)),
    transform = cross(arg1=c("a", "b"),
                      map(arg2=c(1,2),
                          arg3=c("red","blue")))
    )
)
> test_plan
# A tibble: 2 x 2
  target command           
  <chr>  <expr_lst>        
1 test_a c("a", arg2, arg3)
2 test_b c("b", arg2, arg3)

This one works:

test_plan = drake_plan(
  test = target(
    print(c(arg1, args23[1], args23[2])),
    transform = cross(arg1=c("a", "b"),
                      args23=c(c(1,"red"), c(2,"blue")))
    )
)

But the target names are ugly:

> test_plan
# A tibble: 4 x 2
  target            command                                       
  <chr>             <expr_lst>                                    
1 test_a_c.1..red_  print(c("a", c(1, "red")[1], c(1, "red")[2]))…
2 test_b_c.1..red_  print(c("b", c(1, "red")[1], c(1, "red")[2]))…
3 test_a_c.2..blue_ print(c("a", c(2, "blue")[1], c(2, "blue")[2]…
4 test_b_c.2..blue_ print(c("b", c(2, "blue")[1], c(2, "blue")[2]…

What's the best way to do this?

wlandau commented 3 years ago

For static branching (transform = ...) you can use the .data argument of map().

library(drake)
grid <- tibble::tribble(
  ~arg1, ~arg2, ~arg3,
  "a", 1, "red",
  "a", 2, "blue",
  "b", 1, "red",
  "b", 2, "blue"
)
drake_plan(
  x = target(
    f(arg1, arg2, arg3),
    transform = map(.data = !!grid)
  )
)
#> # A tibble: 4 x 2
#>   target     command          
#>   <chr>      <expr_lst>       
#> 1 x_a_1_red  f("a", 1, "red") 
#> 2 x_a_2_blue f("a", 2, "blue")
#> 3 x_b_1_red  f("b", 1, "red") 
#> 4 x_b_2_blue f("b", 2, "blue")

Created on 2020-12-22 by the reprex package (v0.3.0)

The newer targets ecosystem makes this friendlier and more obvious. tarchetypes::tar_map() always takes an argument grid like this, and targets itself supports composable patterns in dynamic branching.

potash commented 3 years ago

Ah, thanks that does it. Also thanks for referring me to targets, I just started using drake so maybe I'll switch over