robbibt
commented
2 years ago Since posting this issue, I have discovered that the drift method gets me closer to what I want, as I can pass in multiple lat/lons as well as multiple times, and avoid the multiple NetCDF reads:
import pandas as pd
from pyTMD import compute_tide_corrections
# Input data (multiple times per point)
example_times = pd.date_range("2022-01-01", "2022-01-30", freq="1D")
point1_df = pd.DataFrame({'lat': -32, 'lon': 155, 'time': example_times})
point2_df = pd.DataFrame({'lat': -33, 'lon': 157, 'time': example_times})
point3_df = pd.DataFrame({'lat': -34, 'lon': 161, 'time': example_times})
# Combine into a single dataframe
points_df = pd.concat([point1_df, point2_df, point3_df])
# Model tide heights using 'drift'
out = compute_tide_corrections(
x=points_df.lon,
y=points_df.lat,
delta_time=points_df.time.values,
DIRECTORY="FES2014",
MODEL="FES2014",
EPSG=4326,
TYPE="drift",
TIME="datetime",
METHOD="bilinear",
)
# Add back into dataframe
points_df['tide_height'] = outHowever, because I have static points with multiple timesteps at each, drift still ends up being less efficient than I want because it assumes each time also has a unique lat/lon, which causes the spatial interpolation step to be run for every individual lat/lon/time pair (rather than being interpolated once for each unique point location then re-used for each time given that the point coordinates are the same for all times).
I think for this application (many timesteps for a smaller set of static modelling point locations), the most efficient processing flow might be something like this?
- Only once per entire analysis: read NetCDF tide model data
- Only once per static modelling point: Extract and interpolate constants based on lat/lon
- For every time: Model tide heights based on extracted constants at each point
(or alternatively, perhaps some method to detect duplicate/repeated lat/lons, then batch those together to reduce the number of required interpolations...)
tsutterley
First of all, congrats @tsutterley on an incredible package... such an amazing resource! I've been looking into using
pyTMDfor modelling tide heights fromFES2014for our DEA Coastlines coastline mapping work. Essentially, our current process is to:I've been testing out the
compute_tide_correctionsas a way to achieve this, passing in the lat/lon of a given 2 x 2 km grid point, and all of the times from my satellite datasets:This works great, but it's pretty slow: about 38.4 seconds in total for a single point lat/lon. Because I can have up to 100+ lat/lon points in a given study area, this will quickly blow out if I want to apply
compute_tide_correctionsto multiple points.Using
line_profiler, it appears that by far most of this time (e.g. 38.2 seconds, or over 99%) is taken up in theextract_FES_constantsfunction:Profiling
extract_FES_constants, it seems like by far the most amount of time in that function (37.5 seconds) is taken up byread_netcdf_file:So essentially, loading the FES2014 files with
read_netcdf_fileoccupies almost all of the time taken to runcompute_tide_corrections. For analyses involving many timesteps for a single lat/lon this isn't a problem, as the files only have to be read once. However, for analyses wherecompute_tide_correctionsneeds to be called multiple times to model tides for multiple lat/lons, the FES2014 data has to be loaded again and again, leading to extremely long processing times.Instead of loading the FES2014 files with
read_netcdf_fileevery timecompute_tide_correctionsis called, could it be possible to give users the option to load the FES files themselves outside of the function, and then pass in the loaded data (i.e. hc, lon, lat) directly to the function via an optional parameter? This would allow users to greatly optimise processing time for analyses that include many lat/lon tide modelling locations.