det-lab / jupyterhub-deploy-kubernetes-jetstream

CDMS JupyterHub deployment on XSEDE Jetstream
0 stars 1 forks source link

support G124 barium analysis #67

Open pibion opened 2 years ago

pibion commented 2 years ago

@MusaabFaozi is working on a G124 barium analysis and they are interested in trying this analysis on the XSEDE-hosted Jupyter system. I think this might be a nice use-case because he's processing raw data and the Dask server might be helpful!

He would need access to G124 data - we'd need someone on the CDMS side to work on copying this data to the OSN so it's accessible. I'll work on identifying someone who can help with this.

He'll also need a high-memory node. He could start with the 16GB node that's available, but it's possible he'll need more memory.

zonca commented 2 years ago

JupyterHub profiles

Here is the current configuration, that I haven't yet updated for Jetstream 2:

https://github.com/det-lab/jupyterhub-deploy-kubernetes-jetstream/blob/9b095af2b38d24011c3e0e85ef9cfb790c144620/config_standard_storage.yaml#L37-L51

I am currently using "m3.medium" instances for the worker nodes, they have 30 GB of RAM, so assuming we can use 25 GB, I think we can do:

I also want to remove all CPU limits, so numbers above are garantueed CPUs, but if there is availability, they can potentially use all 8 vCPU.

pibion commented 2 years ago

@zonca I think 25 GB RAM is a great place to start.

zonca commented 2 years ago

ok, it is deployed, as usual please open an issue with the error log if anything is broken.

Let's keep this issue open to track the barium analysis work.