marbl-ecosys / HiRes-CESM-analysis

Notebooks and tools for validating the 0.1 degree POP / CICE run with ocean BGC
http://hires-cesm-analysis.dokku.projectpythia.org/Interactive_Dashboard
5 stars 7 forks source link

Add interactive notebook example and update environment #44

Closed mgrover1 closed 3 years ago

mgrover1 commented 3 years ago

Adds an example of the interactive dashboard and updates the environment to include additional dependencies. Closes #27.

mnlevy1981 commented 3 years ago

I'm a little out of the loop, so a couple of questions as I try to catch up. I've looked at the notebook in nbviewer but haven't checked out a local copy to play with yet.

  1. How does this tie in to https://github.com/andersy005/HiRes-CESM-dashboard? I noticed that you added Procfile and its one line contains the --allow-websocket-origin=hires-cesm.herokuapp.com option; is your notebook piggybacking on the active dashboard running in a CISL VM, or will this eventually replace what's running in the VM?
  2. I see the requirements file now includes git+https://github.com/andersy005/panelify.git -- should this get moved out of @andersy005's personal space? Will it eventually end up in the NCAR or pangeo organization?
andersy005 commented 3 years ago

How does this tie in to andersy005/HiRes-CESM-dashboard?

This PR implements everything in andersy005/HiRes-CESM-dashboard. So, we should be able to deploy a live dashboard from this repo.

I noticed that you added Procfile and its one line contains the --allow-websocket-origin=hires-cesm.herokuapp.com option; is your notebook piggybacking on the active dashboard running in a CISL VM, or will this eventually replace what's running in the VM?

We should pick a name under which to deploy the live dashboard, and then we can update this to --allow-websocket-origin=$NAME.dokku.projectpythia.org

I see the requirements file now includes git+https://github.com/andersy005/panelify.git -- should this get moved out of @andersy005's personal space? Will it eventually end up in the NCAR or pangeo organization?

Happy to transfer this to NCAR or any other appropriate GitHub org.

mnlevy1981 commented 3 years ago

How does this tie in to andersy005/HiRes-CESM-dashboard?

This PR implements everything in andersy005/HiRes-CESM-dashboard. So, we should be able to deploy a live dashboard from this repo.

That's awesome! We've talked in the past about how we will eventually split this repository up across multiple repos (perhaps a general CESM analysis backend to house CaseClass, with some of the general POP analysis functions moving to pop-tools, and the high-res specific comparisons build to rely on those packages). I like keeping the dashboard here for now, but long-term I think it will end up in it's own repo without the HiRes moniker.

I noticed that you added Procfile and its one line contains the --allow-websocket-origin=hires-cesm.herokuapp.com option; is your notebook piggybacking on the active dashboard running in a CISL VM, or will this eventually replace what's running in the VM?

We should pick a name under which to deploy the live dashboard, and then we can update this to --allow-websocket-origin=$NAME.dokku.projectpythia.org

That sounds good to me. I don't think there's any particular rush, I was just curious - but I think this would be a good future PR.

I see the requirements file now includes git+https://github.com/andersy005/panelify.git -- should this get moved out of @andersy005's personal space? Will it eventually end up in the NCAR or pangeo organization?

Happy to transfer this to NCAR or any other appropriate GitHub org.

Again, there's no rush... but I think we do eventually want it in an org

mgrover1 commented 3 years ago

@mnlevy1981 sure - I can change the example to include the other run. Thanks for the information about the different runs.

mgrover1 commented 3 years ago

@mnlevy1981 I noticed that the https://webext.cgd.ucar.edu/g.e22.G1850ECO_JRA_HR.TL319_t13.004/ plots include the following plot types:

mnlevy1981 commented 3 years ago

Per our review of #42, if you update to the latest master then local copies of the CSV files are stored in the repo (./notebooks/images/${CASE}/png_catalog.csv). This could be useful for creating a notebook that reads data locally rather than reaching out to webext (although it would be nice to include an example of the latter as well)

mgrover1 commented 3 years ago

Where are the plots stored now? Should I be fetching the images from webext then? Did we decide on not storing them within this repo?

mgrover1 commented 3 years ago

Alrighty - I finally got back to working on this, finding the issue with running the dashboard on the JupyterHub. For now, the only way to get the dashboard to work directly with the Glade filesystem is to use Jupyter Forward

Here is an example of the command I used from my local computer, which setups up the forward, port, and the conda environment I am using. This will pull up the JupyterLab interface in your browser

jupyter-forward mgrover@casper.ucar.edu --port=2222 --conda-env /glade/u/home/mgrover/miniconda3/envs/hires-marbl

I submitted a ticket with CISL to add the necessary plugin required to fully interact with the dashboard through the typical JupyterHub interface.

andersy005 commented 3 years ago

@mgrover1, @mnlevy1981, the environment is fixed... Feel free to merge this at your earliest convenience