MilesMcBain / tflow

An opinionated lightweight template for smooth targets flows.
Other
89 stars 9 forks source link

Use Rstudio jobs panel to run tar_make()? #3

Open joelnitta opened 3 years ago

joelnitta commented 3 years ago

Just an idea I had on a run :)

The RStudio jobs panel is convenient because we can see the usual tar_make() output as it runs. With the "Run a targets pipeline in the background" the usual output of tar_make() isn't shown, although we can monitor with tar_progress().

Right now I have a script with a single line, targets::tar_make() that I run with the RStudio job scheduler, but it would be more convenient to have a tflow add-in that does this. The RStudio job scheduler involves a remarkable number of clicks to select and run one script :/

MilesMcBain commented 3 years ago

Sorry I haven't used RStudio jobs. So it can run addins? And the issue is that the addin already in {targets} doesn't give you the output you'd like to see?

Is that true also for the Run a targets pipeline in the foreground addin?

joelnitta commented 3 years ago

Rather than using Rstudio jobs to run the add-in, I would like to the add-in to run (call) Rstudio jobs (have no idea if that's possible though!)

What I've noticed with Run a targets pipeline in the background is that I can't see what's going on or when it's done, or how to kill the job if it hangs for some reason. The Rstudio jobs panel shows the output as it's running in the background, so I think it might be quite useful for making targets.

If you want to see what I mean, create a script with targets::tar_make() and save that as e.g. make.R. Then with that script open, click on the "jobs" tab in the console panel (next to "terminal"), select the script, and run it. (Having the script open just lets the jobs console automatically select it for you, so you don't have to click through directories to find it).

jobs

Run a targets pipeline in the foreground is basically like calling targets::tar_make() in the current session: it shows the output live, but (obviously) runs in the foreground, so you can't execute any other R code while you're waiting for it to finish.

ginolhac commented 3 years ago

Hi,

I am using the trick to use a custom Build script, so tar_make() runs in an independent job, don't blur the console, you can use tar_watch() and you get the nice keyboard shortcut of building a package. See this from my presentation last December:

https://ulhpc-tutorials.readthedocs.io/en/latest/maths/R/PS10_targets.html#19

joelnitta commented 3 years ago

Slick! I had no idea about that particular RStudio functionality.

ginolhac commented 3 years ago

actually, I made a fork of {tflow} to also create the run.R as I am using this by default now. I think the initial trick is from Noam Ross from his {drake} usage but I cannot find the source anymore. Let's acknowledge him anyway