Gridded interpolation speedup

micah-prime commented 5 years ago

Description

This pull request includes the changes required to speedup the local gradient gridded interpolation method. The three main changes are:

Extraction of the Delaunay call in scipy griddata so that it happens once per run rather than every time griddata is called.
Reworking of the input data manipulation using Pandas to be much faster.
Made threading optional in the loadTopo.py when running viewf and gradient. This option is now added to the config file, with default being True.

The second change accounts for most, if not all of the speedup when using cubic interpolation. The first change has a larger effect when using linear interpolation. See Issue #91 for more details on timing.

Fixes #91

Type of change

[x] Bug fix (not really a bug, we wanted a faster implementation of code)

How Has This Been Tested?

This change has been tested in 2 ways:

[ ] Standard unit tests on Travis-CI. Recently added output variable comparisons to catch unintended changes in code.
[ ] Local test over 12 hours on 04/02/2019 in the Tuolumne using the HRRR data. This showed no change at all during the 12 hours for air temp, vapor pressure, and precip mass, as shown below:

gradient_update_check

Test Configuration: Test A:

Docker on Travis-CI Test B:
Python 3.5 on Ubuntu 16.04

scotthavens commented 5 years ago

Should also note in the PR that the loadTopo was modified to provide an option to use threading when calculating gradient and viewf. As of now it is set to False meaning a slight slow down when SMRF initializes.

micah-prime commented 5 years ago

Added those changes to the original PR in the first section. Should we leave threading set to False?

scotthavens commented 5 years ago

We should probably just do it right and add it to the config in the topo section then update the PR.

micah-prime commented 5 years ago

@scotthavens sounds good. I'll do that in a minute

micah-prime commented 5 years ago

@scotthavens That is now added to the config file under ['topo][threading] with default being True. Took a little longer because this upset a check in the backup of the config file. Tests are running on Travic CI now.

USDA-ARS-NWRC / smrf