ProjectPythia / cookbook-template

Project Pythia Cookbook template
https://projectpythia.org/cookbook-template/
Apache License 2.0
23 stars 17 forks source link

Conda environment not found for "trigger-book-build" action #114

Closed bonnland closed 1 year ago

bonnland commented 1 year ago

Hi,

I was interested in submitting a notebook, but I don't understand the workflows well enough to troubleshoot the build error I am seeing when submitting a pull request from my development branch to main on my cookbook template-based repository (https://github.com/NCAR/na-cordex-viz-cookbook).

I'm currently building via "cache" rather than "binder" because it seems that the binder service is timing out. The notebook relies on ~20 dask workers with around 10GB of working memory each, so I'm wondering if this notebook might be asking for too many resources and failing for that reason.

The build has appeared to create the conda environment "na-cordex-viz-cookbook-dev" at least once, but currently when I change the environment.yml file by adding or removing a package, it skips the environment build.

The error I'm getting is this:

building [mo]: targets for 0 po files that are out of date
building [html]: targets for 2 source files that are out of date
updating environment: [new config] 2 added, 0 changed, 0 removed
reading sources... [ 50%] README                                               
reading sources... [100%] notebooks/plot-diagnostics                           
/home/runner/work/na-cordex-viz-cookbook/na-cordex-viz-cookbook/notebooks/plot-diagnostics.ipynb: Executing notebook using local CWD [mystnb]

Exception occurred:
  File "/usr/share/miniconda3/envs/na-cordex-viz-cookbook-dev/lib/python3.11/site-packages/jupyter_client/kernelspec.py", line [28](https://github.com/NCAR/na-cordex-viz-cookbook/actions/runs/4877914656/jobs/8703051137?pr=1#step:17:29)7, in get_kernel_spec
    raise NoSuchKernel(kernel_name)
jupyter_client.kernelspec.NoSuchKernel: No such kernel named na-cordex-viz-cookbook-dev

My current PR with build results is here: https://github.com/NCAR/na-cordex-viz-cookbook/pull/1

It's also possible I was not supposed to check in changes to the "main" branch: I ended up changing the environment.yml, actions environment names, and notebook to run by modifying the "main" branch directly before creating a dev branch for merge.

If I can supply more information, let me know! Thanks.

brian-rose commented 1 year ago

Hi @bonnland!

Building via "cache" means executing the notebooks on the GitHub Actions service, which is unlikely to have the resources to run 20 dask workers. We're actually in the middle of building a new Pythia binder that will better support that kind of workflow, but it's not quite ready yet. We may be able to use your notebook as a good test case (pinging @ktyle about this!)

There's something funny going on with your environment though, which doesn't seem to have anything to do with resources. I'll try to dig in.

brian-rose commented 1 year ago

Investigating in https://github.com/NCAR/na-cordex-viz-cookbook/pull/2#issuecomment-1535289862

ktyle commented 1 year ago

Thanks for submitting this, @bonnland ! As @brian-rose mentions, we do intend to enable Dask cluster capability using our Jetstream2 Binderhub. Right now I'm running through your notebook using a dask LocalCluster on one of my department's servers. I will let you know if I run into any issues!

bonnland commented 1 year ago

Thanks @ktyle! Currently using binder fails on a "git reset" command. I wonder if it could be a permissions problems for an outside user? Just a hunch, though...

brian-rose commented 1 year ago

I've been tasked with shepherding the NA-CORDEX Data Visualization Cookbook through the review process. I'll take a closer look at this issue as part of that process. At the moment I can't recall if there is or is not a problem that needs solving here.

brian-rose commented 1 year ago

Closing this, as the NA-CORDEX Cookbook is building normally now.