NOAA-ORR-ERD / PyGnome

The General NOAA Operational Modeling Environment
https://gnome.orr.noaa.gov/doc/pygnome/index.html
Other
59 stars 44 forks source link

issues running PyGnome #25

Closed MElhagaly closed 1 year ago

MElhagaly commented 7 years ago

Hi all,

After installing PyGnome and getting it working it was time to discover the various possibilities and features within PyGnome. I edited my script((based on code snippets from examples in the script folder) and added a couple of new things to test:

import os
from datetime import datetime, timedelta

import numpy as np

import gnome
from gnome import scripting
from gnome.basic_types import datetime_value_2d

from gnome.utilities.projections import GeoProjection
from gnome.utilities.remote_data import get_datafile

from gnome.environment import Wind, Tide
from gnome.map import MapFromBNA

from gnome.model import Model
from gnome.spill import continuous_release_spill
from gnome.scripting import surface_point_line_spill
from gnome.movers import GridCurrentMover

from gnome.outputters import Renderer, NetCDFOutput, KMZOutput

# # let's get the console log working:
gnome.initialize_console_log()

# define base directory
base_dir = os.path.dirname(__file__)

print 'creating the maps'
mapfile = get_datafile(os.path.join('coast2.bna'))
gnome_map = MapFromBNA(mapfile,
                       refloat_halflife=1,  # hours
                       raster_size=2048*2048  # about 4 MB
                       )

renderer = Renderer(mapfile,
                    "images",
                    size=(800, 800),
                    projection_class=GeoProjection)

print 'initializing the model'
start_time = datetime(2017, 3, 7, 0, 0)

# 15 minutes in seconds
# Default to now, rounded to the nearest hour
model = Model(time_step=900,
              start_time=start_time,
              duration=timedelta(hours=6),
              map=gnome_map,
              uncertain=True)

print 'adding outputters'
model.outputters += renderer

netcdf_file = os.path.join(base_dir, 'test.nc')
scripting.remove_netcdf(netcdf_file)
model.outputters += NetCDFOutput(netcdf_file, which_data='all')

model.outputters += KMZOutput(os.path.join(base_dir, 'test.kmz'))

c_mover = GridCurrentMover('output2.nc')
print 'adding current'
model.movers += c_mover

print 'adding a spill'

spill = surface_point_line_spill(num_elements=1000,
                                 start_position=(5.0, 53.2, 0.0),
                                 release_time=start_time,
                                 end_release_time= start_time + timedelta(hours=6),
                                 amount=5000,
                                 substance='ALASKA NORTH SLOPE (MIDDLE PIPELINE)',
                                 units='bbl',
                                 windage_range=(0.01,0.02),
                                 windage_persist=-1,
                                 name='My spill')

model.spills += spill

scripting.make_images_dir()
print "running the model"
model.full_run()

The console log is activated and I get the following error:

/Users/MustiDarsh/anaconda2/envs/gnome/bin/python2.7 /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/veld.py creating the maps initializing the model INFO - model - line:342 - 25428 - rewound model - Model adding outputters removed /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/test.nc removed /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/test.nc adding current INFO - model - line:342 - 25428 - rewound model - Model INFO - model - line:342 - 25428 - rewound model - Model adding a spill DEBUG - _transaction - line:108 - new transaction DEBUG - element_type - line:71 - 25428 - constructed element_type: ElementType DEBUG - spill - line:711 - 25428 - deepcopied spill 70516c1c-0e5a-11e7-8899-a45e60ebaf6b images_dir is: /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/images removing... /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/images running the model WARNING - element_type - line:98 - obj_type attribute does not exist in element_type initializers INFO - model - line:342 - 25428 - rewound model - Model WARNING - element_type - line:98 - obj_type attribute does not exist in element_type initializers INFO - spill_container - line:498 - 25428 - number of substances: 1 INFO - spill_container - line:498 - 25428 - number of substances: 1 Starting animation <type 'netCDF4._netCDF4.Dataset'> root group (NETCDF4 data model, file format HDF5): dimensions(sizes): variables(dimensions): groups:

<type 'netCDF4._netCDF4.Dataset'> root group (NETCDF4 data model, file format HDF5): dimensions(sizes): variables(dimensions): groups:

DEBUG - model - line:692 - 25428 - setup_model_run complete for: Model Traceback (most recent call last): File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/veld.py", line 85, in model.full_run() File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/model.py", line 968, in full_run results = self.step() File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/model.py", line 915, in step num_released = sc.release_elements(self.time_step, self.model_time) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill_container.py", line 920, in release_elements self._data_arrays) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 800, in set_newparticle_values self._elem_mass(num_new_particles, current_time, time_step) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 459, in _elem_mass _mass = self.get_mass('kg') File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 679, in get_mass water_temp = self.water.get('temperature') AttributeError: 'NoneType' object has no attribute 'get'

I have tried to figure out the problem by going to the files shown in the error, sadly I wasn't able to find what is causing the error. Hopefully someone here could.

I have also tried a different type of spill to see whether the error is within a definition of a spill or something else. The following spill def was tested:

from gnome.spill import continuous_release_spill
end_time = start_time + timedelta(hours=6)
spill = continuous_release_spill(initial_elements=100,
                                 num_elements=1000,
                                 start_position=(5.0, 53.2, 0.0),
                                 release_time=start_time,
                                 end_position=None,
                                 end_release_time=None,
                                 element_type=None,
                                 substance=None,
                                 on=True,
                                 amount=1000,
                                 units='bbls',
                                 name='Point Release')

The following error was printed to my terminal.

/Users/MustiDarsh/anaconda2/envs/gnome/bin/python2.7 /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/veld.py creating the maps initializing the model adding outputters adding current adding a spill Traceback (most recent call last): File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/veld.py", line 81, in name='Point Release') File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 1045, in continuous_release_spill name=name) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 262, in init raise TypeError("Units must be provided with amount spilled") TypeError: Units must be provided with amount spilled

I tried different units like l, bbl, tonnes, tons, ton but the error remained. Chris mentioned that internally mass is used since oil is actually conserved that way. From what I found in the code is that kg is the standard unit for mass in PyGnome. Since it was mentioned that mass is the standard internally I tried to use define different units (g,kg, tons and tonnes) but the error remained.

I found that the script_weatherers.py contained a spill example where barrels were the unit used. When I ran that script I got the following error:

/Users/MustiDarsh/anaconda2/envs/gnome/bin/python2.7 /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/scripts/script_weatherers/script_weatherers.py images_dir is: /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/scripts/script_weatherers/images removing... /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/scripts/script_weatherers/images initializing the model removed /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/scripts/script_weatherers/script_weatherers.nc removed /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/scripts/script_weatherers/script_weatherers.nc adding a spill adding a RandomMover: adding a wind mover: adding weatherers and cleanup options: <type 'netCDF4._netCDF4.Dataset'> root group (NETCDF4 data model, file format HDF5): dimensions(sizes): variables(dimensions): groups:

<type 'netCDF4._netCDF4.Dataset'> root group (NETCDF4 data model, file format HDF5): dimensions(sizes): variables(dimensions): groups:

Traceback (most recent call last): File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/scripts/script_weatherers/script_weatherers.py", line 153, in model.full_run() File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/model.py", line 968, in full_run results = self.step() File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/model.py", line 881, in step "is invalid", msgs) RuntimeError: ('Setup model run complete but model is invalid', ['warning: ChemicalDispersion: no waves object defined'])

To find out whether the error is in my code or in an external dependency I ran the file run_all.py in the /scripts/ directory. After executing a couple scripts the terminal was disrupted by the following error:

/Users/MustiDarsh/anaconda2/envs/gnome/bin/python2.7 /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/veld.py creating the maps initializing the model INFO - model - line:342 - 25428 - rewound model - Model adding outputters removed /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/test.nc removed /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/test.nc adding current INFO - model - line:342 - 25428 - rewound model - Model INFO - model - line:342 - 25428 - rewound model - Model adding a spill DEBUG - _transaction - line:108 - new transaction DEBUG - element_type - line:71 - 25428 - constructed element_type: ElementType DEBUG - spill - line:711 - 25428 - deepcopied spill 70516c1c-0e5a-11e7-8899-a45e60ebaf6b images_dir is: /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/images removing... /Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/images running the model WARNING - element_type - line:98 - obj_type attribute does not exist in element_type initializers INFO - model - line:342 - 25428 - rewound model - Model WARNING - element_type - line:98 - obj_type attribute does not exist in element_type initializers INFO - spill_container - line:498 - 25428 - number of substances: 1 INFO - spill_container - line:498 - 25428 - number of substances: 1 Starting animation <type 'netCDF4._netCDF4.Dataset'> root group (NETCDF4 data model, file format HDF5): dimensions(sizes): variables(dimensions): groups:

<type 'netCDF4._netCDF4.Dataset'> root group (NETCDF4 data model, file format HDF5): dimensions(sizes): variables(dimensions): groups:

DEBUG - model - line:692 - 25428 - setup_model_run complete for: Model Traceback (most recent call last): File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/MMMvM/testPG01/veld.py", line 85, in model.full_run() File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/model.py", line 968, in full_run results = self.step() File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/model.py", line 915, in step num_released = sc.release_elements(self.time_step, self.model_time) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill_container.py", line 920, in release_elements self._data_arrays) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 800, in set_newparticle_values self._elem_mass(num_new_particles, current_time, time_step) File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 459, in _elem_mass _mass = self.get_mass('kg') File "/Users/MustiDarsh/GNOME_SOURCE/PyGnome/py_gnome/gnome/spill/spill.py", line 679, in get_mass water_temp = self.water.get('temperature') AttributeError: 'NoneType' object has no attribute 'get'

For now I can create and execute scripts according to the example here: http://noaa-orr-erd.github.io/PyGnome/scripting/scripting_intro.html#initialize-the-model

I hope these console outputs help in figuring this out. I am going to look in the code and hopefully find what causes the error.

Looking forward to hearing from you.

Regards,

MD

ChrisBarker-NOAA commented 7 years ago

Sorry for the issues -- your original script works fine for me.

But I think I recognise the cause of your errors, and they have been fixed -- I"ll try to get a new version pushed to gitHub today.

A few notes:

If weathering is turned on , then you need a water and waves object added to the model.

and you should be able to run as is with a "non-weathering" elements, that is, don't specify a substance:

spill = point_line_release_spill(num_elements=100,
                                 start_position=(-70.911432,
                                                 42.369142, 0.0),
                                 release_time=start_time,
                                 end_release_time=end_time)

stay tuned for the update.

-CHB

MElhagaly commented 7 years ago

@ChrisBarker-NOAA

Thank you for your reply.

I am sorry to bother you with the issues I encounter while working with PyGnome. I am looking forward to the update.

Appreciate the explanatory notes!

I will stay tuned for you update here, but I would appreciate it I you could let me know as soon as it is pushed.

To run the update when it comes out, I will have to re-clone the repository:

git clone https://github.com/NOAA-ORR-ERD/PyGnome.git

and

reinstall the requirements if necessary

conda install --file conda_requirements.txt\

and rebuild on macOS using:

$ cd py_gnome $ ./build_anaconda.sh develop

and on Windows using:

python setup.py develop

right?

Regards,

MD

ChrisBarker-NOAA commented 7 years ago

On Wed, Mar 22, 2017 at 11:40 AM, MustiDarsh notifications@github.com wrote:

I will stay tuned for you update here, but I would appreciate it I you could let me know as soon as it is pushed.

will do.

To run the update when it comes out,

I will have to re-clone the repository:

The point of cloning the repo is that you have a "live" connection to the code. so you can do:

git pull

and you'll get all the latest code.

reinstall the requirements if necessary

conda install --file conda_requirements.txt\

probably not necessary this time, but good to do.

and rebuild on macOS using:

$ cd py_gnome $ ./build_anaconda.sh develop

and on Windows using:

python setup.py develop

yes, except hat you may want to run "cleanall" before rebuilding.

./build_anaconda.sh cleanall

or

python setup.py cleanall

-CHB

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

MElhagaly commented 7 years ago

@ChrisBarker-NOAA

Thank you for your detailed explaination. Looking forward to it!

Regards,

MD