It might make sense to look into using a quadtree for these seed points to find the closest seed to an arbitrary lat/lon and use the great arc distance to each closest seed point for calculating elevation.
Of course this will not include any filling of sinks though. Not sure how useful this idea will be in the long run.
In order to calculate elevation we use a combination of noise and distance fields (graph distance from mountain, ocean, and coastline seed points). https://www.redblobgames.com/x/1728-elevation-control/
It might make sense to look into using a quadtree for these seed points to find the closest seed to an arbitrary lat/lon and use the great arc distance to each closest seed point for calculating elevation. Of course this will not include any filling of sinks though. Not sure how useful this idea will be in the long run.