mfilter
commented
5 years ago if this solves more problems as it creates I'm fine with that. However I cannot fully anticipate if there are new problems arising due to that. One general comment also relevant for @llavall : The workaround that is selected in FoodAuthent is that their system always include a R server docker that has ALL R libraries included. So this means we should do it with our KNIME Server in a similar way and have by default ALL R libraries installed on the R Server instance. HTH
Last week @tasjabfr could not run the Full NoV model after updating FSK-Lab. Here is why.
Issue
The libraries installed with miniCRAN into the temporary library folder are not used by doParallel. I do not know much of doParallel but I have the impression it is running a number of R processes that do not have any knowledge of the R configuration used in the main process. Therefore, changes to the library paths which are necessary to point to the miniCRAN libraries are not applied in these processes. As a result the model fails when the first worker tries to use the doParallel package. Until now the main process running script has no errors since it knows the miniCRAN libraries.
Maybe @mczyzj and @tasjabfr can comment something on doParallel.
Workaround
The workaround I used last week was to install doParallel manually into the system library. This way the workers can use the package as they know the system library.
Solution
In R a library is a path with installed packages and usually there are two libraries, the system library in C and the library in the user folder. We could consider moving the miniCRAN libraries to the user folder so that the workers would work off the shelve without needing to install the concurrent package manually. We could kill two birds with one stone and use this implementation to avoid deleting the temporary folder and the need to download and install the packages on each start. @mfilter asked me this.