ENSEMBLE_SST

Code for calculation of the FCDR ST ensemble from the AVHRR Easy-FCDR harmonisation parameters and covariance matrix.

Contents

• setup.py - main build script to be run with Python 3.6
• convert_func.py - functions for radiance <--> brightness temperature conversion functions plus measurement equations
• ensemble_func.py - functions for calculation of the ensemble
• inmyvenv.sh - conda environment activation script
• plot_cov2ensemble.py - plotting routines for cov2ensemble calculations
• plot_ensemble.py - plotting routines for ensemble calculations
• run_test_cov2ensemble.sh - submission script for test_cov2ensemble.py
• run_test_ensemble2bt.sh - submission script for test_ensemble2bt.py
• sample_binormal.py - 2D-normal distribution (joint) Monte Carlo sampling test
• test_cov2ensemble.py - generate ensemble given a covariance matrix and parameter mean values
• test_ensemble2bt.py - generate ensemble brightness temperatures given a radiance ensemble

Installation from Sources

We recommend installing ENSEMBLE_SST into an isolated Python 3 environment, because this approach avoids clashes with existing versions of 3rd-party Python package requirements. For example, using (Miniconda or Anaconda) which will usually also avoid platform-specific issues caused by module native binaries.

The first step is to clone latest ENSEMBLE_SST code and step into the check out directory:

$ git clone https://github.com/patternizer/ENSEMBLE_SST.git
$ cd ENSEMBLE_SST

Using Conda

Conda is the package manager used by the Miniconda or Anaconda Python distributions.

To create a new Conda environment ENSEMBLE_SST-env in your Anaconda/Miniconda installation directory, type:

$ conda env create

If you want the environment to be installed in another location, e.g. due to disk space limitations, type:

$ conda env create --prefix some/other/location/for/ENSEMBLE_SST

Next step is to activate the new environment. On Linux/Darwin type:

$ source activate ENSEMBLE_SST-env

In case you used another location use it instead of the name ENSEMBLE_SST. Windows users can omit the source command and just type

> activate ENSEMBLE_SST-env

You can now safely install ENSEMBLE_SST sources into the new ENSEMBLE_SST-env environment.

(ENSEMBLE_SST) $ python setup.py install

Using Standard Python

If you run it with the standard CPython installation, make sure you use a 64-bit version. ENSEMBLE_SST relies on new Python language features and therefore requires Python 3.6+.

ENSEMBLE_SST can be run from sources directly, once the following module requirements are resolved:

• matplotlib
• netCDF4
• numba
• numpy
• pandas
• scipy
• seaborn
• sklearn
• xarray

The most up-to-date list of module requirements is found in the project's environment.yml file.

To install ENSEMBLE_SST into an existing Python 3.6+ environment just for the current user, use

$ python3 setup.py install --user

To install ENSEMBLE_SST for development and for the current user, use

$ python3 setup.py develop --user

There is a known issue on Windows when installing into an existing Python environment. Installation may fail due to an unresolved dependency to the h5py package, which expects pre-installed HDF-5 C-libraries to be present on your computer. You may get around this by pre-installing the ENSEMBLE_SST dependencies (which you'll find in setup.py) on your own, for example by using Christoph Gohlke's Unofficial Windows Binaries for Python Extension Packages.

Getting started

To test the installation, first run the ENSEMBLE_SST command-line interface. Type

$ ENSEMBLE_SST -h

IPython notebooks for various ENSEMBLE_SST use cases are on the way, they will appear in the project's notebooks folder.

To use them interactively, you'll need to install Jupyter and run its Notebook app:

$ conda install jupyter
$ jupyter notebook

Open the notebooks folder and select a use case.

Conda Deployment

There will be a dedicated repository ENSEMBLE_SST-conda that will provide scripts and configuration files to build ENSEMBLE_SST's Conda packages and stand-alone installer.

Development

Contributors

Thanks go to the members of the FIDUCEO project consortium for making the data required available.

Unit-testing

Unit testing will be performed using pytest and its coverage plugin pytest-cov.

To run the unit-tests with coverage, type

$ export NUMBA_DISABLE_JIT=1
$ py.test --cov=ENSEMBLE_SST test

We need to set environment variable NUMBA_DISABLE_JIT to disable JIT compilation by numba, so that coverage reaches the actual Python code. We use Numba's JIT compilation to speed up numeric Python number crunching code.

Generating the Documentation

The documentation will be generated with the Sphinx tool to create a ReadTheDocs. If there is a need to build the docs locally, some additional software packages are required:

$ conda install sphinx sphinx_rtd_theme mock
$ conda install -c conda-forge sphinx-argparse
$ pip install sphinx_autodoc_annotation

To regenerate the HTML docs, type

$ cd doc
$ make html

License

The code is distributed under terms and conditions of the MIT license.

Contact information

• Michael Taylor (https://patternizer.github.io)