Closed jpolton closed 2 years ago
Created a new branch 'feature/met'
Add wiki notes on how to prep 2013/2014 ERA5 forcing data
Added new experiment EXP_barotropicTide_ERA5
that runs with tides and met. See Run tides with met forcing wiki page.
Estuary at i=113, j=114 looks too narrow to let the water easily flow in and out. This leads to a hydrological jump at time step =261, (dt=20s) starting at 1st Jan 2013.
Water in the estuary has a sea level of -5m. At the mouth it is -2m. So there is an unphysical steep SSH gradient at the mouth. This seems to result in a blow up.
FIX: modify the bathymetry to either open the channel, or close the channel in this region.
I will leave this as an exercise for @mpayopayo
@jpolton I'm getting some error (missing parentheses in lines containing print
in OFFICIAL_Generate_NEMO_Forcing_NEWERA.py
) when trying to add atmospheric forcing.
print tout[0], tout[-1], tout.shape, out.shape, LON.shape
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(tout[0], tout[-1], tout.shape, out.shape, LON.shape)?
I'm using the pynemo3 environment because I don't have any other environment-I'm new to python. Can the problem come from there? what packages do you have in your nrct_env
environment so that I can have the same?
@jpolton I'm getting some error (missing brackets in lines containing
OFFICIAL_Generate_NEMO_Forcing_NEWERA.py
) when trying to add atmospheric forcing. I'm using the pynemo3 environment because I don't have any other environment-I'm new to python. Can the problem come from there? what packages do you have in yournrct_env
environment so that I can have the same?
Not sure what happened to my file edits there. The issue with print statements is a version 2 to version 3 thing. python 3 requires brackets: print("This")
. Code SCRIPTS/OFFICIAL_Generate_NEMO_Forcing_NEWERA.py
modified. The conda environment used is vanilla. You should NOT need anything fancy. I just grabbed an old one I had lying around
@jpolton I pulled your branch, so shall i modify the script in my code and then it will be incorporated when pushing changes? still not quite sure of how git works
@jpolton I pulled your branch, so shall i modify the script in my code and then it will be incorporated when pushing changes? still not quite sure of how git works
I don't quite understand. Yes pull the changes from the repo. That should fix the problem you identified. If your changes make it even better than commit them and push it back.
@jpolton I'm getting an error saying the following at the 1st step:
import netcdftime
ModuleNotFoundError: No module named 'netcdftime'
I've tried cloning pynemo3 under a different name to then add that module but I'm getting unexpected errors. Could you please save a txt of your nrct_env environment so that i can create it from that txt file?
conda activate nrct_env
conda list -e
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
basemap=1.0.7=np19py27_0
ca-certificates=2019.6.16=hecc5488_0
cairo=1.12.18=6
certifi=2016.2.28=py27_0
curl=7.45.0=0
cython=0.26=py27_0
fontconfig=2.11.1=6
freetype=2.5.5=1
geos=3.3.3=0
hdf5=1.8.15.1=3
libgfortran=1.0=0
libiconv=1.14=0
libnetcdf=4.3.3.1=3
libpng=1.6.17=0
libxml2=2.9.4=0
libxslt=1.1.29=0
lxml=3.8.0=py27_0
matplotlib=1.4.3=np19py27_2
ncurses=5.9=10
netcdf4=1.1.9=np19py27_0
numpy=1.9.2=py27_0
openssl=1.0.2l=0
pip=9.0.1=py27_1
pixman=0.32.6=0
py2cairo=1.10.0=py27_2
pyjnius=1.4=py27_0
pyparsing=2.0.3=py27_0
pyqt=4.11.4=py27_4
python=2.7.13=0
python-dateutil=2.6.1=py27_0
pytz=2017.2=py27_0
qt=4.8.7=3
readline=6.2=2
requests=2.14.2=py27_0
scipy=0.16.0=np19py27_0
seawater=3.3.4=py_1
setuptools=36.4.0=py27_1
sip=4.18=py27_0
six=1.10.0=py27_0
sqlite=3.13.0=0
thredds_crawler=1.0.0=py27_0
tk=8.5.18=0
wheel=0.29.0=py27_0
zlib=1.2.11=0
I've just tested this python SCRIPTS/OFF...
and it works. However looking at this module list it is python2 not python3! So at some near future point it should be further updated to python3. However, in the mean time, hopefully you can build the nrct_env
.
@jpolton I copied the output of that into a ncrt_env.txt
and tried creating the environment with conda create --name ncrt_env --file ncrt_env.txt
. It fails and I get the following error:
$conda create --name nrct_env --file nrct_env.txt
PackagesNotFoundError: The following packages are not available from current channels:
- ca-certificates==2019.6.16=hecc5488_0
- pyjnius==1.4=py27_0
- thredds_crawler==1.0.0=py27_0
- seawater==3.3.4=py_1
Current channels:
- https://repo.continuum.io/pkgs/main/linux-64
- https://repo.continuum.io/pkgs/main/noarch
- https://repo.continuum.io/pkgs/free/linux-64
- https://repo.continuum.io/pkgs/free/noarch
- https://repo.continuum.io/pkgs/r/linux-64
- https://repo.continuum.io/pkgs/r/noarch
- https://repo.continuum.io/pkgs/pro/linux-64
- https://repo.continuum.io/pkgs/pro/noarch
I tried also specifying the channel
$ conda create -c conda-forge --name nrct_env --file nrct_env.txt
PackagesNotFoundError: The following packages are not available from current channels:
- pyjnius==1.4=py27_0
- thredds_crawler==1.0.0=py27_0
Current channels:
- https://conda.anaconda.org/conda-forge/linux-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.continuum.io/pkgs/main/linux-64
- https://repo.continuum.io/pkgs/main/noarch
- https://repo.continuum.io/pkgs/free/linux-64
- https://repo.continuum.io/pkgs/free/noarch
- https://repo.continuum.io/pkgs/r/linux-64
- https://repo.continuum.io/pkgs/r/noarch
- https://repo.continuum.io/pkgs/pro/linux-64
- https://repo.continuum.io/pkgs/pro/noarch
Any idea of how to sort this? I can maybe do manually the ones that exist in the channel, but how do I get the others?
Another alternative is if you export is as .yml?
It seems that it gives trouble generating environments from the .txt generated with conda list -e
https://github.com/conda/conda/issues/10636
This is the information about the environment in case it helps:
$conda info
user config file : /login/marpay/.condarc
populated config files : /login/marpay/.condarc
conda version : 4.4.10
conda-build version : 3.4.1
python version : 3.6.4.final.0
base environment : /packages/lmodmodules/apps/anaconda/3-5.1.0 (read only)
channel URLs : https://repo.continuum.io/pkgs/main/linux-64
https://repo.continuum.io/pkgs/main/noarch
https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
package cache : /work/marpay/marpay-conda/pkgs
envs directories : /work/marpay/envs
/login/marpay/.conda/envs
/packages/lmodmodules/apps/anaconda/3-5.1.0/envs
platform : linux-64
user-agent : conda/4.4.10 requests/2.18.4 CPython/3.6.4 Linux/3.10.0-1062.12.1.el7.x86_64 centos/7 glibc/2.17
UID:GID : 18725:18063
netrc file : None
offline mode : False
@jpolton when manually I have the same issue. pyjnius==1.4=py27_0
and thredds_crawler==1.0.0=py27_0
are not available in the current channels.
what channels have those packages?
@jpolton I tried also doing pip install pyjnius
and said succesfully installed pyjnius
then did conda install -c conda-forge thredds_crawler
and it also said succesfully installed
.
However when doing conda list -e
the package pyjnius
does not appear while thredds_crawler
does appear. I've tried also doing both import pyjnius
under a python terminal and import jnius
under a python terminal and both give errors:
>>> import pyjnius
File "<stdin>", line 1, in <module>
ImportError: No module named pyjnius
>>> import jnius
File "<stdin>", line 1, in <module>
File "/work/marpay/envs/nrct_envt/lib/python2.7/site-packages/jnius/__init__.py", line 43, in <module>
from .reflect import * # noqa
File "/work/marpay/envs/nrct_envt/lib/python2.7/site-packages/jnius/reflect.py", line 20, in <module>
class Class(with_metaclass(MetaJavaClass, JavaClass)):
File "/work/marpay/envs/nrct_envt/lib/python2.7/site-packages/six.py", line 808, in __new__
return meta(name, bases, d)
File "jnius/jnius_export_class.pxi", line 120, in jnius.MetaJavaClass.__new__
TypeError: Error when calling the metaclass bases
type() argument 1 must be string, not unicode
@jpolton I did pip install pyjnius
and conda install -c conda-forge thredds_crawler
and it doesn't give any error, in both cases it says succesfully installed
. but when doing conda list -e
the package pyjnius
does not appear. When I try to import pyjnius
under a python terminal it also says no module named pyjnius
. And when trying to import jnius
I get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/work/marpay/envs/nrct_envt/lib/python2.7/site-packages/jnius/__init__.py", line 43, in <module>
from .reflect import * # noqa
File "/work/marpay/envs/nrct_envt/lib/python2.7/site-packages/jnius/reflect.py", line 20, in <module>
class Class(with_metaclass(MetaJavaClass, JavaClass)):
File "/work/marpay/envs/nrct_envt/lib/python2.7/site-packages/six.py", line 808, in __new__
return meta(name, bases, d)
File "jnius/jnius_export_class.pxi", line 120, in jnius.MetaJavaClass.__new__
TypeError: Error when calling the metaclass bases
type() argument 1 must be string, not unicode
it doesn't seem to need those modules though. The alternative to import pyjnius
would be to do import autoclass
The python script SCRIPTS/OFFICIAL_Generate_NEMO_Forcing_NEWERA.py
runs fine with env nrct_env
(python2) but you need to change to python3 to do python create_LSM.py
(pynemo3 works fine).
I also made slurm script for the last step and will modify wiki
ONce the bathymetry is modified and the Taw estuary is closed, it ran for a month (January2013) without blowing up. The wiki is modified accordingly
@mpayopayo @jpolton the wiki about the creation of met-forcing (Download ERA5 forcing data 1. Extract ERA5 data) is a bit unclear, with the mention to python2, python3 and python environment (source activate nrct_env). I've managed to extract the ERA5 files and I've updated the wiki with the instructions for a very simple python environment to use (with python3 and I deleted the notes about python2 and 3). I'll need to update just a couple of lines in OFFICIAL_Generate_NEMO_Forcing_NEWERA.py and push it.
@mpayopayo @jpolton the generation of the met forcing can work also with the PyNNEMO environment, I added that in the wiki. I hope I didn't mess anything in OFFICIAL_Generate_NEMO_Forcing_NEWERA.py, it works for me...
Experiment set up with example met forcing (ERA5: pressure and winds) + tides