Verify Great Lakes HPC access and budget with DeepSensor demo

[DaniJonesOcean]

Issue Description:

This task is designed to confirm your ability to access the Great Lakes HPC resources and use our shared computational budget. You'll be running a DeepSensor demo using a Jupyter notebook. While I suggest using the web-based Open OnDemand portal for convenience, you are welcome to use other methods if you prefer.

Steps:

1. Request Access to the Computing Budget:

   • Send @DaniJonesOcean your umich uniqname. They will add you as a subaccount user to the dannes0 subaccount. This should give you access to our shared computing budget.

2. Accessing Great Lakes via Open OnDemand:

   • Head to the Great Lakes User Guide and follow the 'Getting Started (Web-based Open OnDemand)' section to log in. Here's the primary task list:

     • Turn on your U-M associated VPN if necessary
     • Use Duo authentication with your U-M credentials.
     • Navigate to the Great Lakes OnDemand web interface, and log in.
     • Familiarize yourself with the interface and available tools.

   • Great Lakes OnDemand Portal: Getting Started Guide

3. Running the DeepSensor Demo:

   • Once in the Open OnDemand portal, launch a Jupyter notebook server following the provided instructions in the user guide.

   • Consult the 'Getting Started' sections in the DeepSensor documentation and execute the steps therein. Be sure to:

     • Install the DeepSensor package (a pip install usually works fine)
     • Run the provided example commands in the 'Getting Started' guide to ensure that everything is operating correctly.
     • Note any potential issues with package installations or running the notebook.

   • DeepSensor Demo Notebook

   • DeepSensor Documentation: Getting Started

Deliverable:

• A successful run of the sample DeepSensor Jupyter notebooks. Check against the expected output in the 'Getting Started' documentation. Make note of any mismatches.
• A short reflection on the ease of access, any hurdles encountered, and how they were overcome or if further assistance is needed. This will help us with any early access issues.

Note: (I'm assuming that you haven't used Jupyter notebooks, which could be totally wrong! My apologies if I've forgotten.) The Jupyter notebook is a user-friendly option that doesn't require extensive command-line knowledge, but should you choose to work directly via SSH or another interface, let me know!

Completion Criteria:

You will have demonstrated that you can access the HPC system, execute jobs within our assigned budget, and interact with the DeepSensor package, which will be critical as our project progresses. Please tag @DaniJonesOcean in your completion comment.

[DaniJonesOcean]

@eredding02 I have added your uniqname to the dannes0 subaccount. You should now be able to access the compute hour budget under that subaccount. We have a lot - 7,315 GPU hours to be used by the end of June! This resource will auto-refill at the start of July.

[DaniJonesOcean]

@eredding02 I've added a DeepSensor demo notebook to this repository. It only covers very basic functions, but it should be enough to get started:

https://github.com/CIGLR-ai-lab/GreatLakes-TempSensors/blob/main/notebooks/01_dcj_DeepSensor_QuickStart.ipynb

[eredding02]

@DaniJonesOcean I was able to successfully run the DeepSensor demo via UM HPC. Specifically, it was through a Jupyter notebook on OnDemand. I experienced some difficulty running the demo through the HPC rather than locally, but it was resolved through pip installing deepsensor and pytorch. We are currently trying to find a way to have a consistent environment so that this will not be necessary each time we use the HPC.