search

paciorek / future-kubernetes

Instructions for setting up and using a Kubernetes cluster for running R in parallel using the future package.
39 stars 10 forks source link

R, parallel, and kubernetes on PRP-Nautilus #4

Open wsphd opened 3 years ago

wsphd commented 3 years ago

Chris,

Your code and documentation for using R and kubernetes is useful in a variety of contexts.

If you want to experiment with more of a Research/Education cluster, rather than a commercial cloud provider, you might want to investigate using the PRP-Nautilus cluster. UC Berkeley most certainly has access to that (academics-only) cluster. Ryan in your Dept./Colllege or someone else on campus should be able to set you up with a Nautilus namespace. After that, you can created k8s pods, etc. If you can't get access, I can create a test namespace for you too (I have admin. rights on Nautilus).

The more we test and document our efforts with kubernetes and R, the better off everyone will be. Also, since there are no chargebacks for anything, you can run any number of tests (including for deep learning, Bayes, big data, GPU/CUDA tests and times) without worrying about any charges.

Again, thanks for your good work with parallel computing and R. The computation and storage is only becoming more and more computationally-intensive.

Best,

Wayne Smith (wayne.smith@csun.edu) CSU Northridge

paciorek commented 3 years ago

Hi Wayne, thanks for this pointer. I've been aware of Nautilus but hadn't thought of trying R+k8s there.