Use `rpy2` in runner

[dshemetov]

Using rpy2 to call R lets us:

• catch exceptions
• split complex commands and manage R state

Hopefully this isn't too installation specific and can work well with renv.

[brookslogan]

issue (blocking): using rpy2 does appear to be part of the cause of long/infinite run times. With OPENBLAS-OPENMP enabled, I don't see any additional cores launch. It's paused on an empty progress bar, so I wonder if that's the culprit (and if your evalcast update solves it when parallelism isn't used or progress bars are disabled).

[brookslogan]

Some observations:

• Observation 1: rpy2 + ATLAS didn't work [> 3 hrs, cut off].
• Observation 2: rpy2 + OPENBLAS-OPENMP didn't work [>30m, cut off].
• Observation 3: system + OPENBLAS-OPENMP worked.
• Observation 4: system + ATLAS worked. (And also parallelized! Similar runtime to OPENBLAS-OPENMP. Maybe it's just smarter than OPENBLAS-OPENMP in that it does not try to parallelize when you're already heavily utilizing almost all cores, or maybe it looks for something related to how mc.allow.recursive works.)
• ~Observation 5: rpy2 + disabling bettermc works to forecast. (I just used my snippet; haven't tried evalcast branch yet.)~ [or not. seems to have trouble with second forecaster.]
• Test 6 (TODO): try disabling just progress bars.
• Test 7 (TODO): try other parts of pipeline.

[dshemetov]

Linking this related issue rpy2/rpy2#961.

[brookslogan]

Still testing to try to determine what's actually going on.

• Disabling bmc, I see things seem to get stuck on first ahead for second model.

Buggy debugging:

- ~Trying only running the second model: instead it aborts! But the error message seems to be truncated, complicating debugging; not sure truncation is from rpy2 or something else. [Something else! This seems to match the non-rpy2 output. But I don't understand why my current modifications are generating errors for the second model.] [I think this might be something related to evalcast/rlang/other library versions with/without my renv.]~ --- No, I just tried passing a list to `get_predictions`' forecaster arg instead of a function and was confused by errors.

[brookslogan]

I noted some of our debugging findings here. I would suggest we don't move forward with rpy2 for now, and instead: either check the wait/exit status from os.system, or move to something using subprocess as recommended in linked doc. From an earlier experience: we should also check that we properly raise an error within R when install.packages encounters an issue; I think it might only warn in some cases when it should stop.

[dshemetov]

Closing in favor of #17