search

det-lab / jupyterhub-deploy-kubernetes-jetstream

CDMS JupyterHub deployment on XSEDE Jetstream
0 stars 1 forks source link

GPU access through JetStream? #37

Closed pibion closed 4 years ago

pibion commented 4 years ago

@zonca , @00KevinMac and @farnoush-bk are doing some machine learning work that requires GPU computing.

@00KevinMac has taken a look at XSEDE resources, and allocations on pretty much any of their GPU resources (e.g. Bridges or Comet) would serve well.

I'm wondering if it's possible to access these resources from the JetStream JupyterHub, or if JetStream has GPU VMs.

I'd like to stay away from installing the CDMS software on new systems - although maybe that's a fundamental requirement if we want to use other systems? Our experience within the collaboration of getting a system to work with CDMS software - even if it already supports CVMFS - is that it's a real pain.

So another option is: we use the XSEDE JupyterHub for CPU tasks and send people elsewhere for GPU tasks.

zonca commented 4 years ago

Jetstream doesn't have GPU. I think your best choice is to use Comet. They already have cvmfs installed. Also, they are currently installing a platform which will allow users to launch Jupyter Notebooks. I don't know when it will be ready, I'll notify you once it is available.

Anyway even now you can run Jupyter Notebooks yourself, but it is a but laborious. The only problem is that you have to create XSEDE accounts for each user.

zonca commented 4 years ago

also, Jetstream 2 is coming later this year with GPUs! https://www.hpcwire.com/2020/06/02/jetstream-2-cloud-will-deploy-with-amd-and-nvidia-technology

pibion commented 4 years ago

@zonca oh that is fantastic! I'm happy to volunteer some testers :)

zonca commented 4 years ago

I asked about the schedule, they said Jetstream 2 early operations will be summer 2021