RS-DAT / JupyterDaskOnSLURM

Apache License 2.0
16 stars 3 forks source link

Ersdat course config #26

Closed meiertgrootes closed 1 year ago

meiertgrootes commented 1 year ago

Hi @rogerkuou @cpranav93 @fnattino branch (and draft pull request) contains the configuration adaptations needed to make JDonSLURM work with the reservation SURF has provided. It worked for me this way, with all slurm jobs running on wn-ca-01. I haven't tested dCache yet. Have a look if this works for you

rogerkuou commented 1 year ago

Thanks @meiertgrootes , in general it works! I have put my comments in these issues:

RS-DAT/Project_planning#84 RS-DAT/Project_planning#83 RS-DAT/Project_planning#82

BTW I found running runJupyterDaskOnSLURM.py in an actual WSL terminal (but not in VSCode WSL terminal) partly solves the problem for automatically opening browser. In this case it stops at:

(jupyter_dask) oku-ersdat@ID14204:~/JupyterDaskOnSLURM$ python runJupyterDaskOnSLURM.py --platform spider
Submitted batch job 4891969
Waiting for SLURM output: slurm-4891969.out ...
found
SLURM outputfile slurm-4891969.out present. Retrieving node information
node info not present
ssh -i /path/to/private/ssh/key -N -L 8889:wn-ca-01:8788 stursdat-ouku@spider.surf.nl
server not yet available
server not yet available
server not yet available
server not yet available
server not yet available
[I 2023-04-05 16:06:43.411 ServerApp] Jupyter Server 2.5.0 is running at:
8889:wn-ca-01:8788 stursdat-ouku@spider.surf.nl
http://localhost:8889
Webbrowser with connection to remote jupyter server launched
enter 'end' to stop port forwarding:)
gio: http://localhost:8889: Operation not supported

which keeps the port forwarding opening, but do not start my edge. I can ctrl+click on the link http://localhost:8889 to get it open. I think this should be good enough.

cpranav93 commented 1 year ago

I had a look at it along with Ou and it worked fine. There was some minor issue when combined we could not get all 8 cores that were allotted which was odd. Might be something to figure out at a later point. The configuration works great!