nptscot / npt

Data processing code, also use this repo for issue tracking for the Network Planning Tool. See https://nptscot.github.io for development version
https://www.npt.scot/
GNU Affero General Public License v3.0
5 stars 0 forks source link

Use `tar_map()` to reduce duplicated code #374

Open Robinlovelace opened 6 months ago

Robinlovelace commented 6 months ago

From the targets manual:

Sometimes, a pipeline contains more targets than a user can comfortably type by hand. For projects with hundreds of targets, branching can make the _targets.R file more concise and easier to read and maintain.

Just saw this tip from @andrewheiss (thanks for sharing):

Finally figured out how to dynamically make {targets} targets through metaprogramming (https://books.ropensci.org/targets/static.html#metaprogramming) and it's SO NICE to have nice clean target names that are automatically generated #rstats

Source: https://fosstodon.org/@andrew@fediscience.org/111689215513772257

Currently our code has a lot of bits like this:

https://github.com/nptscot/npt/blob/f1ae9ebdae23854361be7e660cc18cce68b14774/_targets.R#L322-L328

An iterator like tar_map() could compress those 3 targets into a single function call, reducing complexity and easing readability and maintenance.

Robinlovelace commented 6 months ago

I tested this on a modified version from the documentation:

  list(
    tarchetypes::tar_map(
      list(a = c(12, 34, 1), b = c(45, 78, 9)),
      targets::tar_target(x, a + b),
      targets::tar_target(y, x + a, pattern = map(x))
    )
  )

And the results show that the 'target factory' is accurate:

image

This can be part of a wider refactor discussed, will do some tests first but I think we can greatly simplify things as per #343 but taking it to the next level.

Robinlovelace commented 4 months ago

Taking a look at this now.

Robinlovelace commented 4 months ago

Heads-up @mem48 and @wangzhao0217 these are the regions I'm planning to use to geographically batch the build process, enabling all routes to be held in a single list:

image

Robinlovelace commented 4 months ago

image