OpenDrift announcements

[knutfrode]

To stay informed about new releases and important changes of OpenDrift, please click "Subscribe" in the menu on the right. A GitHub account is needed.

[knutfrode]

The OpenDrift repository is now moved to https://github.com/OpenDrift

[knutfrode]

A new mechanism for configuration is implemented in https://github.com/OpenDrift/opendrift/commit/10d3579e5c7d7e3a957c125c4bb82c0537939dcd and https://github.com/OpenDrift/opendrift/commit/7c70e36892c10421636554aa16565af2488fb26e.

This involves a slightly modified OpenDrift API:

Earlier, configuration was modified by directly altering ConfigObj items (dictionaries): >>> o.config['drift']['scheme'] = 'runge-kutta'

The same is now achieved with function calls: >>> o.set_config('drift:scheme', 'runge-kutta')

Advantages with the new configuration mechanism include:

• Configuration items are now inherited from superclasses (parent modules), avoiding duplication
• An error is raised in case of an invalid configuration setting, for better integrity

The possible configuration settings for a given model/instance is shown by: >>> o.list_configspec()

The resolution of the Basemap coastline could previously be specified when creating an instance, e.g.: >>> o = OpenOil3D(basemap_resolution='i') This is no longer possible, but should be done using the general configuration mechanism: >>> o.set_config('general:basemap_resolution', 'i')

[johannesro]

This is really a step forward! I have been worried that errors were not raised before, this will really help I think.

Greetings Johannes

Johannes Röhrs, PhD Norwegian Meteorological Institute www.met.no / www.yr.no

johannes.rohrs@met.no +47 41654886

On Fri, Jan 13, 2017 at 11:13 PM, Knut-Frode Dagestad < notifications@github.com> wrote:

A new mechanism for configuration is implemented in 10d3579 https://github.com/OpenDrift/opendrift/commit/10d3579e5c7d7e3a957c125c4bb82c0537939dcd and 7c70e36 https://github.com/OpenDrift/opendrift/commit/7c70e36892c10421636554aa16565af2488fb26e .

This involves a slightly modified OpenDrift API:

Earlier, configuration was modified by directly altering ConfigObj items (dictionaries):

o.config['drift']['scheme'] = 'runge-kutta'

The same is now achieved with function calls:

o.set_config('drift:scheme', 'runge-kutta')

Advantages with the new configuration mechanism include:

• Configuration items are now inherited from superclasses (parent modules), avoiding duplication
• An error is raised in case of an invalid configuration setting, for better integrity

The possible configuration settings for a given model/instance is shown by:

o.list_configspec()

The resolution of the Basemap coastline could previously be specified when creating an instance, e.g.:

o = OpenOil3D(basemap_resolution='i') This is no longer possible, but should be done using the general configuration mechanism: o.set_config('general:basemap_resolution', 'i')

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/OpenDrift/opendrift/issues/32#issuecomment-272562465, or mute the thread https://github.com/notifications/unsubscribe-auth/AITeq3TrSCbnuthZkB7_f8ISgLMhyECMks5rR_cXgaJpZM4K_ESr .

[knutfrode]

Given the new configuration mechanism announced above, a generic method has now been implemented in https://github.com/OpenDrift/opendrift/commit/784df415dcbac49023fcc4dc01a9e15bfc42e41d for interaction with coastline.

Previously, all elements were deactivated when hitting land (except for WindBlow model). Now some alternatives are possible though a common configuration setting:

>>> o.list_configspec()
>>> ...
general:coastline_action [stranding] option('none', 'stranding', 'previous', default='stranding')

To deactivate elements hitting land (default for most models): >>> o.set_config('general:coastline_action', 'stranding')

To ignore coastline (e.g. for atmospheric transport): >>> o.set_config('general:coastline_action', 'none')

To move elements hitting land back to their previous position in water (default for pelagicegg): >>> o.set_config('general:coastline_action', 'previous')

More sophisticated interaction is planned for implementation later, e.g. keep exactly at coastline, slide along coastline etc.

[kristokv]

Hi Knut-Frode,

Do these changes require adjustments in the module file's "configspec" section? It seems like this section is no longer read? For example, I have this section in my module file:

configspec = ''' [drift] scheme = string(default='euler') [processes] turbulentmixing = boolean(default=True) verticaladvection = boolean(default=False) [biology] constantIngestion = float(min=0.0, max=1.0, default=0.5)

'''

Now I get the following error:

KeyError Traceback (most recent call last)

in () ----> 1 o.set_config('biology:constantIngestion', 0.75) /Users/kristina/Documents/opendrift/opendrift/models/basemodel.pyc in set_config(self, key, value) 244 ds = self.configobj.configspec 245 for i, s in enumerate(key.split(':')): --> 246 if not isinstance(d[s], dict): 247 d[s] = value 248 else: /Users/kristina/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/configobj.pyc in __getitem__(self, key) 552 def __getitem__(self, key): 553 """Fetch the item and do string interpolation.""" --> 554 val = dict.__getitem__(self, key) 555 if self.main.interpolation: 556 if isinstance(val, six.string_types): KeyError: 'biology'

[knutfrode]

I am sorry, but the documentation is not yet updated properly on this. In your case, you can instead of the configspec-string add this line to your __init__() method: self._add_config('biology:constantIngestion', 'float(min=0.0, max=1.0, default=0.5)') The other config-items will be inherited from the superclasses. In your update() method, the actual value can be retrieved with: self.get_config('biology:constantIngestion')

Alternatively, a configspec-string with several items (like before) may be created in your constructor as a local variable, and added with a single command: self._add_configstring(configspec_local) Thus self.configspec will not be propagated as before.

[kristokv]

Thanks for the quick reply? With the first option, the "self._add_config(..." should be added within def init(self, *args, **kwargs): ?

I get the following error doing this: TypeError: _add_config() takes at least 4 arguments (3 given)

[knutfrode]

Oh yes, and there also has to be a comment/helpstring, e.g. to be shown within a graphical or WEB interface. This could probably have been made optional, but for now it is compulsory, e.g.: self._add_config('biology:constantIngestion', 'float(min=0.0, max=1.0, default=0.5)', comment='Ingestion constant')

[knutfrode]

There are now two major updates of OpenDrift, as developed by André Brodtkorb (Sintef) within a project funded by Uninett Sigma2.

• The Basemap landmask checking is significantly faster. It uses a rasterised map for a first screening of elements which are far from coast. Results are identical as before. Using a full resolution landmask is now feasible, although initialisation may take some time (~one minute).

• The default interpolation method was previously 'nearest neighbour', as the linear interpolation was very slow, at least for longer simulations. There is now a new interpolation method ('linearNDFast'), which is much faster. This is now the default option. Although still not as fast at 'nearest' interpolation, another benefit is that e.g. missing data between an ocean model and the basemap coastline is interpolated from the nearest valid value, preventing offshore stranding for cases where no other driver data (e.g. wind) is used.

Se here for more details.

[knutfrode]

Some new improvements are available in the latest version (1.0.3) of OpenDrift:

• Horizontal interpolation is faster, and extrapolation of ocean model data towards coastline is improved.
• A new scheme for vertical mixing (Visser et al. 1997) has been implemented by Johannes Röhrs. Tests indicate that the vertical distribution (oil droplets, fish eggs, etc) is improved, and is about 7 times faster than the old scheme.
• Oil weathering calculations are now also faster.
• The netCDF reader now supports using ensemble input.
• A reader is available for netCDF unstructured grids (but needs further improvement).
• Animations now support comparing many simulation runs (compare=<list of other simulations>), and also coloring of the elements according to any element property (color=<property>).
• Several other minor improvements and bugfixes.

[knutfrode]

Version 1.0.6 of OpenDrift includes several improvements:

• Improvement of performance, including e.g.:
  • multiproccessing for interpolation
  • a bug in SciPy interpolation routines has been acounted for. Interpolation of unprojected readers is now much faster.
  • Initialisation of readers is now delayed until the reader is needed (LazyReaders), for better performance (if desired)
  • ROMS_native_reader is faster by caching and reusing sigma-to-z-coefficients
  • This reader may now also aggregate netCDF4 files (MFDataset) if xarray is installed (which is not supported by netCDF4-python)
• Some basic functionality for Lagrangian Coherent Structures, see example scripts for demonstration
• A module for plastics: PlastDrift
• Runge-Kutta 4th order advection scheme is implemented
• If a (long) simulation crashes, it is now possible to import and analyse the incomplete netCDF file with OpenDrift

[knutfrode]

Important information to OpenDrift model developers: (@gauteh, @johannesro, @magnesim, @trondkr, @manuelaghito, @simonweppe, @John-Luick, @AndresSepulveda, @vic1309 ....please subscribe to this ticket (right menu) of you want to get noticed about such changes in the future)

As you might have seen, OpenDrift has undergone some fundamental changes from version 1.3.3 to 1.4.2. Whereas these are clear improvements for the future, some updates are necessary for existing models to continue working. These changes have been implemented for all models in the repository, but you might need to apply the same to your own "external" models:

1. Previously models defined a list of required_variables (CF standard names), and corresponding lists for desired_variables (which are not strictly required). Furthermore, a dictionary fallback_values defined the values to be used for these variables, if not provided by any readers. The above is now all collected in a dictionary called standard_name. For each variable one can add:

   • "important": True/False, where False (deafault if keyword importantis omitted) is the same as listing the variable in desired_variables
   • "fallback": <value> for fallback values. If fallback values is None, the simulation will stop if there are no other readers which can provide this variable. Here is an example of how this is implemented in the OpenOil model.

2. A new internal configuration mechanism, omitting the previously used configobj-package. The user API (set_configspec(key, value), value = get_configspec(key), ...) is mostly unchanged, but configuration options are now defined in the models in a a different way than before, with a dictionary provided to the method _add_config. This should be called in the constructor (init) of your model, but after the parent constructor has been called, see example from OpenOil. One of the new things here, is the keyword description, providing auto-generated documentation, e.g. for the GUI.

The configuration mechanism is now also used to set fallback values, which is safer than manually editing the dict fallback_valuesas before, as there is now a check that variables/keywords really exist, and that values are allowed (e.g. between min and max). The fallback-values have keys like environment:fallback:<variable>, e.g.:

>>> o.set_config('environment:fallback:x_wind', 5)

Correspondingly one can now also replace fallbackwith constant, to provide a constant value to be used for the simulation (overriding any other available readers):

>>> o.set_config('environment:constant:x_wind', 5)

One final note: as OpenDrift from time to time get new dependencies, it is necessary to update your conda environment after updating your code with git pull:

$ conda env update -f environment.yml

Among new dependencies, are Xarray/Dask, which is used for some new functionality to load huge output files lazily, in order to facilitate post-processing of netCDF files too large to fit in memory.

[John-Luick]

I did a general update (git pull) on 27 October. Looking at /opendrift/models/oceandrift.py, it does not seem to have a version number in it. What is the easiest way to tell what version one is running? (I'm a bit of a git neophyte.)

[gauteh]

Closing, new releases announced here: https://opendrift.github.io/history_link.html.