Jasmin access, virtual machines, jupyter etc

ESMValGroup / ESMValTool_Tutorial

ESMValTool Tutorial

https://tutorial.esmvaltool.org/

Other

5 stars 8 forks source link

Jasmin access, virtual machines, jupyter etc #16

Open ledm opened 4 years ago

ledm commented 4 years ago

Last week, we also discussed setting up virtual machines, using a ready-made conda on jasmin, and we have also discussed a jupyter notebook. These are supposedly easy way for people to get started on ESMValTool at face-to-face tutorials without having to install ESMValTool.

None of this appears in the tutorial at the moment, so it would be good to set something. I don't know anything about that though. Can someone volunteer to write something in 02-installation.md? @valeriupredoi :beers: if you do.

The second question is: this is useful for a face to face set up, but less so for a DIY try out ESMValTool yourself at home setting. I guess we should include both options in the installation page, but be clear that one is for tutorials and one if for "real" use.

valeriupredoi commented 4 years ago

a VM is also useful for personal user use too, if they just want to muck around or even if they want to run actual pre-made recipes; I think we should aim at a central installation on Jasmin, rather than a VM, accessible via module load - a VM can be created anyway and we can host the VM box somewhere for the user to load it on the go and get into it (and that should contain data too) but on HPC's there is not much of a point deploying VM's when we have all the CMIP data just a few dirs away

ledm commented 4 years ago

Okay, if thats the plan for all tutorials, will Jasmin access be a problem? Isn't it an UK/EU based machine?

Either way, could you let me know how it would work and I'll start writing it into 02-installation.md.

valeriupredoi commented 4 years ago

I meant aVM is not needed on an HPC, I still think setting up a VM and uploading it on a shared platform for users to download it is a very good idea. In my vew the plan would be this:

once we have 2.0 stable we deploy it on HPC's (I will arrange that for CEDA-Jasmin, and it will be avaialbale cluster-wide via eg module load esmvaltool; we need peeps to do the same for DKRZ, etc)
we st up a VM that contains esmvaltool installed in a functioning environment and some CMIP and OBS data, upload it somewhere (ideas where?), so people can load it in virtualbox and play around - the same way as @glad64 has done for the Singapore workshop
we set up a docker+singularity container so this could be deployed on various locations (a bit more advanced - @bouweandela @jvegasbsc and myself are onto it)

How's it sound?

bouweandela commented 4 years ago

I believe Javi came up with the idea of using virtual box for the training. Maybe he already has content for the installation part of the tutorial? @jvegasbsc would you be interested in contributing that?

jvegreg commented 4 years ago

Maybe he already has content for the installation part of the tutorial? @jvegasbsc would you be interested in contributing that?

I don't have content but I can help with that

sverhoeven commented 4 years ago

I have some experience with Virtualbox:

In a KNIME graphical workflow engine tutorial I used packer to create a Virtualbox based virtual machine which could be started with Vagrant. The image was hosted on https://app.vagrantup.com/boxes/search .
In a tutorial for Xenon where we (Xenon developers) created a VM manually and uploaded it to Zenodo.

For the ESMValTool tutorial we could use packer to create Virtualbox/Vagrant images automatically.