environmental-modeling-workflows / watershed-workflow

Python workflows for data-rich, hyper-resolution simulations of hydrologic models on watersheds.
Other
67 stars 29 forks source link

create_osx_envs.sh fails for 1.4 #66

Closed rfiorella closed 1 year ago

rfiorella commented 1 year ago

create_osx_envs.sh invokes function arguments in create_envs.py that have changed as of ec3802d9010cb3d7455dd3880f0eefef71d5204c.

Tried updating from:

ENV_NAME="watershed_workflow-`date +%F`"
python environments/create_envs.py --manager=mamba --env-name=${ENV_NAME} --with-user-env --user-env-name watershed_workflow_user --with-tools-env OSX

which fails because --env-name, --user-env-name arguments have been removed and --OS added - to:

ENV_NAME="watershed_workflow-`date +%F`"
python environments/create_envs.py --manager=mamba --with-user-env watershed_workflow_user --with-tools-env watershed_workflow_user --OS OSX ${ENV_NAME}

which issues an error:

Traceback (most recent call last):
  File "/Users/rfiorella/repos/watershed-workflow/environments/create_envs.py", line 222, in <module>
    create_and_dump_env_local(args.env_type, args.OS, packages, args.name, args.dump_only)
AttributeError: 'Namespace' object has no attribute 'name'
ecoon commented 1 year ago

Right, that args.name needs to be args.ENV_NAME.

Also, I don't think you can make the tools and user env the same name. It will try to build two different environments there. We should probably add a check to confirm that all three of tools, user, and ww envs are unique names.

Effectively, "tools" is just to get you an environment to build ExodusII without polluting your main environment. It includes gcc/gfortran, netCDF, etc which will probably break most of your system compilers/mpi/ATS install. Once you've used tools to build ExodusII, you probably don't actually want to "activate" that environment ever again.

The "user" env is the "default" environment. You don't have to use this, but I find it easier to have a "default" environment that has Jupyter and some basic packages. This is the one you have active all the time, and from which you call jupyter lab. Then the main WW env is the one used as a kernel once you start the notebook. Not having all the extra Jupyter stuff in that environment makes this a bit easier to solve and have the most recent Jupyter (in user) while having some older packages for shapely (in WW we're still stuck with shapely < 2.0).

Also, I need to document the create_envs script a bit more with this description!

rfiorella commented 1 year ago

Thanks - that clarifies why tools and user are separate.

A couple quick follow-up questions: Is tools env necessary if you already have compilers/mpi though something like brew or macports instead? For exodusII, is there any implied install location? Or just the binaries/libraries need to be in the appropriate environment variables? (e.g., already have a compiled version for TINerator.)

(As an aside - David and I have been talking a bit about the shapely issue for TINerator. I'll send you a separate email but we were curious about how you're thinking about dealing with this for WW to see if there's a consistent approach across both packages that might work).