search

ropensci-review-tools / pkgstats

Historical statistics of every R package ever
https://docs.ropensci.org/pkgstats/
17 stars 1 forks source link

Check sys instead of system for ctags/gtags calls #18

Closed mpadge closed 2 years ago

mpadge commented 2 years ago

Following on from ropensci/roweb3#236, and this comment from @jeroen.

mpadge commented 2 years ago

@jeroen swapping system for sys::exec_wait() improves things enough to make parallel computation feasible. It still fails because of nodes suffering "error reading from connection" that causes a whole-system dump. My guess there is an error attempting to read from the files produced during the exec_wait() calls, for which there is no signal sent from the process to actually wait for. The whole thing is nevertheless chunked with intermediate saves, and using sys makes it stable enough to run in parallel, and just restart when system dumps. system would never even make it through a single chunk.

Great confirmation for the stability of sys over system - thanks!

Update: Nope, looking at the results, the parallel job still doesn't work - the failed nodes dump entire arrays of junk values which end up being most of them. The process itself is more stable, but the results end up just as useless. I'll use sys regardless, but the analyses will have to remain single-threaded.