search

richfitz / remake

Make-like declarative workflows in R
Other
340 stars 32 forks source link

Best way to call remake::make() from a PBS script? #59

Closed wcornwell closed 9 years ago

wcornwell commented 9 years ago

any ideas?

richfitz commented 9 years ago

Ah, HPC systems. Such a joy to use.

If you run 

remake::install_remake(".")

on the head node it will put some scripts in the working directory. Then your pbs script can include:

./remake

as the command and everything should work.

wcornwell commented 9 years ago

Mostly works, except I'm having working directory issues.

Does remake.yml have to be in the same folder as the remake scripts?

Can you set the working directory at the top of the remake.yml file and then have all the subsequent paths be relative?

richfitz commented 9 years ago

Afraid so. But I can set you up a change directory on the command line tool if that's useful. I'm avoiding thinking about some deeper issues with working directories as it's much harder than it seems at first glance.

wcornwell commented 9 years ago

I've got this to work, now I think

wlandau-lilly commented 7 years ago

@wcornwell I know this is an old issue, but if you want to split up remake targets over multiple PBS jobs, another option is parallelRemake/remakeGenerator using the Makefile and shell.sh in described here: that is, if the overhead is worth your while.

wcornwell commented 7 years ago

@wlandau-lilly cool thanks. I'll check it out.